{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pinecone-io/examples/blob/master/learn/search/faiss-ebook/product-quantization/pq-simple.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/pinecone-io/examples/blob/master/learn/search/faiss-ebook/product-quantization/pq-simple.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Simple PQ Implementation\n",
    "\n",
    "In this notebook we will implement product quantization (PQ) using simple, readable, Python code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = [1, 8, 3, 9, 1, 2, 9, 4, 5, 4, 6, 2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first step is the creation of `m` subvectors:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = 4\n",
    "D = len(x)\n",
    "# ensure D is divisable by m\n",
    "assert D % m == 0\n",
    "# length of each subvector will be D / m (D* in notation)\n",
    "D_ = int(D / m)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[1, 8, 3], [9, 1, 2], [9, 4, 5], [4, 6, 2]]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# now create the subvectors\n",
    "u = [x[row:row+D_] for row in range(0, D, D_)]\n",
    "u"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we must create a set of clusters for each subvector space - giving us `m` seperate *codebooks* (codebook will map our subvectors to their assigned cluster centroids - *reproduction values*).\n",
    "\n",
    "The clusters would usually be trained, we will not do that here as this example is using only one vector. We will use randomly generated centroid positions.\n",
    "\n",
    "We need to decide how many centroids create - more centroids == lower error between vector positions and the centroids they are assigned to (more centroids increases the chances of vectors being assigned to a closer centroid).\n",
    "\n",
    "This value is chosen by `k`, which must be divisable by `m` to create equal (sub)centroid ranges for each subvector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "k=32, k_=8\n"
     ]
    }
   ],
   "source": [
    "k = 2**5\n",
    "assert k % m == 0\n",
    "k_ = int(k/m)\n",
    "print(f\"{k=}, {k_=}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have `32` centroids in total, and `8` centroids per subvector space (subspace).\n",
    "\n",
    "Each of these centroids will have three dimensions - aligned to our subvector dimensionality. Let's generate them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from random import randint\n",
    "\n",
    "c = []  # our overall list of reproduction values\n",
    "for j in range(m):\n",
    "    # each j represents a subvector (and therefore subquantizer) position\n",
    "    c_j = []\n",
    "    for i in range(k_):\n",
    "        # each i represents a cluster/reproduction value position *inside* each subspace j\n",
    "        c_ji = [randint(0, 9) for _ in range(D_)]\n",
    "        c_j.append(c_ji)  # add cluster centroid to subspace list\n",
    "    # add subspace list of centroids to overall list\n",
    "    c.append(c_j)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a lot of centroids in here so the easiest way for us to *see* them is to visualize:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATUAAAD9CAYAAADd0+BpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABo2klEQVR4nO29d5gcV53v/a3OYbpnOk0OGmmUNVHBNggbjA04jmzLJi3gCy+YsO8FlvXCJexlL7CwL2u4YC5hubAGe1lAGhlkW8YRs9jYSJalSZogTc6dp3NXV9V5/xidck9P5+4JGtfnefw8ltShuuvX3/M7v3QYQggkJCQkNguy9b4ACQkJiWIiiZqEhMSmQhI1CQmJTYUkahISEpsKSdQkJCQ2FZKoSUhIbCokUZOQkNhUSKImISGxqXjDiRrDMO9jGGaCYZggwzC/YxjGvN7XJCFRKAzDVDEMc5JhmFmGYQjDMFvW+5rWizeUqDEMsxfATwB8AEAFgBCAH67rRUlIFAcBwB8A3LXeF7LeXPGixjBMHcMwJxiGcTAM42IY5gdpHv5+AI8RQv6LEBIA8BUAdzIMY1ibq5WQyJ5cbJsQskAI+SGAM2t4iRuSK1rUGIaRA3gcwASALQBqAPw6zVP2AuimfyCEjABgAexYvauUkMidPGxb4jKK9b6AAjkEoBrA/YQQ7vLfvZjm8SUAFhP+bhGA5KlJbDRytW2Jy1zRnhqAOgATcTc9EwEAxoS/MwLwF/WqJCQKJ1fblrjMlS5qUwDqGYbJ1uPsB9BK/8AwzFYAagDDq3BtEhKFkKttS1zmShe10wDmAHyLYRg9wzAahmHenObx/wHgNoZh3sIwjB7A/wJwghAieWoSG41cbRsMw2iwtEgDgPryn99wXNGiRgjhAdwGoAnAJIBpAO9O8/h+AB/HkrjZsRRL++TqX6mERG7katuXCWMpxAIAg5f//IaDkSbfSkhIbCauaE9NQkJCIpFNJ2oMw/yYYZhAkv9+vN7XJiFRCJJtZ4e0/ZSQkNhUZEoXS4q3cWDW+wI2GZJtbxyKatubbvspISHxxkYSNQkJiU2FJGoSEhKbCknUJCQkNhWSqElISGwqJFGTkJDYVFyREwAIIRAEATzPQ6FQQCaTtFlic0AIAcdxYBgGcrkcDCNV8uTKFSdq9KZHo1GwLAuZTAaZTAalUgmFQgG5XC6JnMQViSAIiMViCIdf70PnOA5arRZqtVoSuSy5okSN3nRBEJatZIQQRKNRRKNRAJBETuKKghACnucRi8UALNkvteuRkRFUV1fDYFgazqxQKKBUKiGXyyWRS8EVIWrxN51hGMhkMgiCAABgGGbZjSWErBA5uVwuGoJCoZAMQWLDQAgBy7LiQk0IwYQ7hEl3BGqlDDJuSeTkcrn4O+C414fhUpGjYRjJtq8AUSOEIBaLgef5ZQKW6uYlEzlBENDT04PKykoYjUZR5KgnJxmCxHogCAJYlgUhRLTbUWcIL416YdTIwQWBubkIaqt5GJHctuNFjmEYKBQK8b83qshtaFFLdtNzhT6PbleplxeJRMTHSCInsZbQuDBNCMSHR4bm/bDolFArZWDAYEpg4PBHUWVd+TrJRI7jOHEb+0YVuQ0paulueqGk8uQkkZNYCxK3m4m2JZMxEOJa7QlS70oSSWbbsVhshcjRUMxmFbkNJ2qZbnqxSSVy4XBY/PtgMAiLxSKJnERBcByHqakpVFZWprTt5mojnhtYQJhjwPMEepUM5SXKvN6P7k4oyUQuMaG2GWx7Q4laoutczC8427lx9H2pd0gIweDgIDo6OsTriXfpJZGTyET8zmN8fBzV1dUpH1tj0uKGXVbMeMJQKWXgtH6oFcXZqSQTOZZlEY1GYbfbYTKZUFJSItr2WjgVq8GGELXE7WY2XyTP8+B5PqetaSExOYVCIV5rfCYWkNLsEqlJLEPKBmuJCha9EgzD4KJflvWCnCvxIufxeGA0GkWRA14vjaK2faWI3LqLmiAI8Hg8mJ+fx7Zt27L60nw+H3p7ewEsxb/KyspgNpthNBpXvSYtmScnpdklEklWe7aRIYSIpSP0zwDAsixYlgWw9BkSY3IbkXUTtfibzvM8QqFQxh8/IQRTU1OYmZlBc3MzVCoVOI6D1+vF/Pw8hoeHoVarYTKZRFd6PWJy8SLncDhQVVUlidwbiFRlSPlAa9dWG1phEP++AK5IkVsXUUu86QqFQiymTQXHcejv74dcLsehQ4cAALFYDCqVCuXl5SgvLwcAhMNheDweTE5OIhAIQK/Xw2QygeO4NTGORCOenJyEzWaTaoneIBSjDGk9EAQhrShdSSK35qKW7KZnWo38fj96e3uxZcsWMcjK83zSx2q1Wmi1WlRXV4MQgmAwCI/HA5/Ph/7+fpSVlYmenFqtTvoaxSZTBuqNkGbf7KxWGdJ6eWqZSCZyNPEQL3Lr0a64ZqKW7qbHtz0lMj09jcnJSbS0tECr08MbikEuY6BTZr4BDMOgpKQEJSUlCAQCqKmpASEEHo8HFy5cAMdxKC0tFUWOJgMSr7uYZJNm/8lPfoL7779fErcrhNUsQ4p/rcVwDK+MeRBieeyrNmJ7ub5o75OrqCWSTbvi5OQkhoeHcfTo0YKvNx1rImqZCw5XihrHcbhw4QIA4NChQ+AIg+cG7XAGYiAg2GbVoaVKl9NAOIZhYDQaUVpaii1btoDneSwuLsLj8WBiYgIMwyxLOtB+u0JvdqZ/TxS5X/3qV7j//vvzfk+JtYPGhXPZbmayqUTvjBACf4TDA8+OwhWMQS4Hnh1y4f95Ux3a60qL8jkKtfNEkoncyMgI/vKXv1z5opbNTU8UtUAggJ6eHtTX16O2thYAcG7cA0+IQ2WpGgIhGF4IwKKVocGS/2oll8thNpthNpsBLMXovF4v7HY7Ll26BIVCgbKyMvA8X/SbngnJS9vYEELg8/kQCoVgMpmy3lpRwcqlS4AQgp4ZH5xBFtVlGgBAIMrhsV570UQtl5KTfGAYBsFgECUlJav2HpRVE7VcYgzxojY7O4vx8XE0NzeL41YAwB1kUaJZ8mhkDAO1Qo5AlEv6evmiVCphs9lgs9kAANFoFE6nEyzL4vTp09DpdOJWVafTScLzBoXWnvl8Png8HnFRzAaZLL+6s5hAlh2OqZAxiPHpk2u5QEs6VpNAIHDlilpiwWE2WzBBENDX1wee53Ho0KEV8S1riRrD9gC0SjkEgYDleZRq82sfyRa1Wo3y8nLY7Xa0tbUhHA7D7XZjdHQUoVAIBoNBFDmNRrPi+fkYL8uyUKlUxbh8iSKTWHuWTdY+EWrr8SGHRKLR6LJCbkIIdleUQK2QwR2MQaVgsBjicEdbZUGfJ9m1rSahUOjKE7XEm55tjCEUCiEYDKKurg51dXXJe+JqjPBFYphfjIAA2FdtQJVx9bOX8dtmnU4HnU6H2trapTiH3w+Px4PBwUGwLLss6aBUKjOmyZNBy1AkNhbJas/kcnleopZusZuensb4+DgAiNl5mUyGOpsNn7l+Kx7rXUCQ5fHO3TZct92S9+dZD4LBIKzWJONGikzRRC3fgsO5uTmMjY1Bq9Wivr4+5eNUChneusOKEMtDLmPACJzozq7mCpMq/kGTDkajEQ0NDRAEQUw6TE1NgRACo9EotnOlW5njWSsXXSJ7UtWeyWSylKVFqUiV6ed5HgMDAxAEAQcOHADDMIhEIrh06RLsdjvm5uZgMBhw186lGPCV6M0Hg8E1WbCLImr5FBzyPC96OAcPHsSZM2cyPodhGOjVCvj9fvT09ECtViMcDsNgMIgB/2Lf7GyDujKZTPTSgKXs7czcAk6+uoDvnXsZVq0MH9hvw/bachgMhpQe3Fq56BKZyRQXTleKlIpkzwmHw+ju7kZVVRXq6+vBcRwEQYBWq4XRaIRGo0F5eTn8fj/cbrcYpqH2VlZWlvWiuZ4Eg8FlcfLVoiBRSzZmOxtCoRB6enrEm5iLp0UTCS0tLVAql5p+42+2IAhiWUZpaWnBNzvfrJBCocC/n/fhLwtAiVYNl0/Av7zkwReuZsFHglCr1TCbzTCZTNDr9cvGHOl0uoKuWaJwsqk9k8vlOXtqidtPp9OJoaEh7NmzR1wQkz0+fmdAy5G8Xq8Y41UoFOLCvhbtgfmw4T21fAsO5+fnMTIygn379qG0NPt0tCAIomd36NAhyGQysCy74mbTXlCn04mRkREolUqYzWaxrCSfz5lPVigc4/GX8UWUKBlolXJolXIEWR6csRpXtZjEpMP4+LiY6jaZTFhYWJA8tXUm29qzfD01Wpg6OjoKt9uNAwcO5NzdIpfLYbFYYLEsxdWi0SjcbrfYHkjtyWw2J01irQcbuqRDEATMzMyA4zhUVVVlJWiCIGBoaAjhcBiHDh2CUpl95jISiaC7uxvl5eXYvXu3mEFK9r4KhQJWq1UMSEYiEbjdbgQCAfT396O0tDSnrWq+9WkKGQMGAE3EE0IAQiCXLf1Zq9WipqZG7HIIBoNwu9145JFH8Oqrr+Kd73wnbr755pzfVyJ/cm11ytdTY1kWQ0NDKCkpwf79+zP2XGazGKvValRVVaGqqmqZPQ0ODiIWi4l2X1ZWlrRzZi3YsNtPOsiR53lEIpGss5v04JNdu3blJBJutxsDAwPYtWuXuCrlgkajQXV1NRYXF1FTUyO+ZnxcIt1WNV9RU8pluGmXCY/1O8FHOAiEYItFh+Zq44rHxrdz3XHHHbjqqqtw7bXX5vyeEvmTz84jH0+NDmbYvn07KiuzL8kghGDMFcK8LwqjRoGdFSVQypOLYbw91dfXi50zdGdA47+0c2attqobuqSDNmFns0rZ7XZcvHgRe/fuRVlZWdrHxgsIIQQTExNYWFjA/v37C3ah6WsbDIaMW1Wz2SzGuQrpJPibDitKZVHYBT2qjGrc2VaZcYppMBhEaWmptAVdY+KHK+RS7Z8LMzMz8Hg82LVrV1pBSxwBRAjBa1M+nJ/2QaeSIxrjMeON4oZdVtHzT0di5wzLsnC73ZidncXg4CB0Oh1YlkU4HIZWq83pM+XChi2+pVMk5HL5ssGIiQiCgOHhYQSDQRw8eDDjVi9eQOhqplAocPDgwaJOPIgn1VaVxrkMBgM0Gk3Oq7EIIbiusQRbt27N+imBQABVVVX5vZ9EQazWqKD4eHBFRUXO8TNOIOib9aHSqL4sYkrM+SLwhGKwluSe7VepVKisrERlZSUIIQiFQjh//jyGh4cRjUZX1FsWi2g0uiaTcfLeXKeLJ4TDYfT09MBms2Hnzp05ufL0uXV1dWLf51pBt6p0bJHf78fs7Cw8Hg/OnDmTcauaSD7Ft6FQSCq+3UTQeHBFRQV2796N4eHhnBJWDMNA4Mnl/1/+b8WYIEOLylUqFVpbW1fUWwJYtlUt1MFYi/FDeYuaQqFI6qk5HA4MDw+nTFGnQiaTwW63Y2xsLKvMaDqhXIo/hOENx2ArUaHOlLtLTbOqtHB269at8Hq9cLlcKbeqieQjamuVIZJYSbG9NJfLhcHBQezevVvc+uUTh1PKGewo12NgPgCDRoFQjINVr4JJX5yazPgQS2K9ZSwWE8ftDw8PQ6PRiCKXS//zWg6EyFnU4ofDxXtqgiDg4sWL8Pv9WW034yGEIBwOY3p6OufnJnutU/0OPD/khEwGEALc0VqJ7OV15evR7XZ8Cj0SiYgjiwKBgNgHajabRRdbErU3JoQQjI2Nwel0rogHZ5vNjC8nEQQBh7aUwahVYMEXxVatFnurDFBkEU/L9npTCY5SqVw2WToUCsHj8WB0dHRZ4bvJZMr4u10rYSvK9jMSiaCnpwcWiwX79+/P6cJZlkVvby8YhsHevXuzFjSv14vZ2VlxVaHbQWeAxQsXXagsXYo/xHgBJ3sX8J6t+bnqqW6ERqNZlkIPBAJwu93i8MmysjIIgpDzVlIStSuPeBuJxWLo6+uDRqPBgQMHknYh5Lr9BAC5jMHeKgP2VhW/JCIXsaH9zzU1NRAEQSx8n5mZSVv4ns8Cny8FixqtiM6n5MLn86Gvrw9NTU2YnZ3N+nkzMzOYnJxEXV0dPB4PxsbGxO1gSK4HA4hZIaVcBkIAli+uqMVDs6oGgwENDQ1itff4+DhcLhccDgcsFktWh8FIbVLrRz5eBN1OyuVycex8Y2NjymQP9bxyYbXHeefbNSOTyVBaWorS0lI0NjYuqya4dOkSVCqVGKLJZ4HPl7y3nwzDIBwOY2xsLK+KaCpMra2t0Ov1mJ+fz3izaQFvNBrFgQMHwPO86BZHIhG4XC4E7DMILvoxHlTCVqpDgJOh3KCGXhXJ9aMCyM9lpltVn8+HkpISGI3GFdXe9GYnfm9SQ/uVBV3cFxYWxPa9dPcv15jaWmzXijVLLV01wUMPPYTZ2Vk888wzuPHGGwt+r7TXkc+TotEoenp6AECcKJAt8entgwcPitXNmW52LBZDd3c3TCYTdu3aJb4WRaPRiBX6VVtC+MVfxjHtDsKsjOFApRqhIC9mFnO53kLiANTlTqz2TrZVpdXea1V1LVEcGIbB0NAQOI5LOgcw2eNz9bxW21NbrVhXfDXBRz/6UVy8eHFj1qkJgoBz586hqakJw8PDOX0Zient+OemEzU63rupqUn0zNLd6BqTDl+8ZY94s6LRKPr6+jA7O4uxsTEYjUZxO5ipDqcYohZPqq2q2+3GqVOnMD09jX/7t3/D5z73ubzeUyJ/cr3PkUgEi4uLqK2txb59+7IuXcqltWotPLXVHuUNLJV51dbW4pprrlnV9wHyLL696qqrwDAMhoeHs34ebXeKT28nvm4yoaLnBWRy65NBb5RarRbnten1evh8PrhcLkxOToJhGJjNZlgsFhgMhhU3t9iilkh8VnX79u344Q9/KHqiEhsXas9GoxHV1dU5dSGslqfWN+vHE30LUCvluLu9CjVl2XXhrMUo77Wa0AHkuf2kAkSDnum+kGzbnRIDqHSKgcfjwYEDB4o2J41hGDG4uXXrVrFlZHp6Gn6/HyUlJbBYLGLDezaiRghBOCZAIWOgimuDyjXjQwiBQqGQGtk3MIQQjI+Pw263Y//+/RgZGcnJ88omphYMBsXK/mxF8OUxD/77b/sR4QTIGOA3Z2fx2490ZFWjuRalFhte1OgXTYOkqX64HMehr68PKpUqY7tT/M2mz9NoNOjo6FjVVSSxZSRxNptSqYRer09540Msj39/eQp9c37IGAa3NZfjnbttWQl+KjbiLKw3Apm+92T2nE/gP51IxU+CHhoaglKphEqlQiwWSxsq+e7zY4hwS9chECAY5fHLv07jS+/anvGa1krU1ipWXNAMEipqyb7sYDAoHnNHp2OkI75N6vz581k9L9MBFrmSOJstFovh0qVLWFxcxOnTp6HX60UvjmYtH+2eR++cH1UGNXiB4NHuedSUatBcY8zLU5PYmNC47pYtW1BdXS3+fa7jh1KJICEEly5dgs/nw/79+8VOlqmpKTidTjExR0MViaVBYXb5NRAsCVs2SNvP+CenaJWikzmam5thNK4ctZMMmUwGn8+HkZGRjBM9CCEQBEH8D8CqHGuvVCpRUlKCsrIyVFZWIhgMwuVyiVlLs9mMnskQTFrF0uQSOQMFI8OEO5yXqEWj0Q0z0E/idebn5zE6Orri2EYgvxKNxMWL53n09vZCo9Ggvb1dHOctCALUajXKysqWhUpoaRBNeJnNZtzWUoGfvjSJSGzpWjQKGW7ZV57VNa1FomAti8rz3n4CK1cpQkjerVJerxd+vx+HDh1K+8OOFzSVSgVBEMTDTei10EkixRC5+HYVOqOqoaEBHMfB4/FAx3hxaWERFp0CKpUaUY6BRb/kuabbmidDOklqfUn8YdNJM6FQCAcPHky6IynUU4tEIjh//jxqa2tRU1MjHlxEt5x2ux1lZWXiyHzassQwzLKEV4eGwZ27DHh2LAS1Qo6/va4Bb96W3Xmka7X9zGV+XCEUZfsJLLU79fT0oLS0FB0dHVl/SYIgYGBgAJFIBA0NDRkFLf60dCpc9PxFKnj0mqiBFCJuqW64QqGAzWbDJ2404rvPj8IbZLEYYlGr4wHHJVzkTWBZNqf3kkRt4xCNRtHd3Q2LxYL29vaijfSO99S8Xi/6+/uxZ88eGI3GZRN3aftgZWWl2JJEbZvaZElJCQwGA7Zu3YpoNIqaajdu2+pCKBSCUeGBw7HUnB5fOydcfm9Z3OdZK1Hb0J6a+OTLgyIXFxfR19eH7du3i3Vk2UANp7y8HEajMa1xJBO0eKhwyeVy8cxN+niO4xCJRMBxXM5Cl+mGlxvU+MpNOzDpDkOlkKHBrAWIAK/Xi/n5eZw7dw5qtVrcJqQ7VEVqkVp/GIYRC6N37tyZ8ZzKXM/+pCI4OzuLiYkJtLe3Q61WL5u4GwgExPZB+v7UXuMX73ihk8vlqKioQFVVFQRBEL248fFx8VCWIZ8CL04EQAhwTaMJ79xjg1yWejR+MVlL2y7YU7Pb7fD5fGhra8vJy/D5fOjt7RUNZ3Z2NuXQSZ7nxS8+WzGKz0wNDg7CaDSipKRENMB4cctUkpLphutUcuyqjL9hS7VnarUahw4dQjgchsvlwqVLlxCJRMQOgvhGfGBtg6kSK6HlGnNzc+jo6MhqCqxMJhMP784Wr9crtvpRG6WC5nK5cPHiRezbty+pCMQv3gBEUYsPwwAQJzxv27YNkUgEfxmawYlzszAqeeg0GjzTH4ZeJcN1O6xrkihYy11I3jE1QRBgt9tBCFnW7pQNc3NzGB8fXyaEyYpv41ekfKaSxmIxcVhl/EHJ8V4cNQRaopIonPm65rxAMLbIw9m3AJ1Kjva6CtTW1kIQBLGDIL4R32KxSNvPDYBcLhdPK8uGXLafHMfh4sWLIISgra1thW1PTU1hfn4eHR0dWcej4xflVCEYhUKBRaJDlc0Ci16JSCSCyGIQfzw3jLLwDFQqVV6HsYRjS++hVWauQNjw289wOIzXXnsNWq0WpaWlWX8hhJBlI77jn5doHIUKGp2g29jYuGJLnMwQ6GoHLBkffUy+otY948OgW0CzjcAbjmLB78BtzRXQqZbPi6dNv6Ojo/jBD34At9uN/v5+7N27N+f3lCgMhmFQX1+f03Yy20QBLVWihxJTu6O2NTQ0BJZl0dHRkXeZUroQjFEtQ5jlIGjlUGs0UMVkaKkzYseOMnGajMfjgclkgsViQVlZWUph5wWCV8Y9uLQQBABsryjBVVvK0p6XsOHr1CKRCLZv3w6WZREMBrN6DvWajEZj0sBrfEdBpvhZJnw+nxiAzTRBN5M7H41GRW8ul+3vwHwAJg0DnUoOnUqOeV8UzgCLevPyLU180++dd96J7u7uVT38QqK4ZOOpxScENBoNzpw5g7m5OVitVshkMvT29sJoNGLHjh0rbD3GCzjZs4D5xQj21Rhx3fbsx3vFL95varJhwB7GrDcMQoBSrQKHG41QKpXiQT/V1dXLRgdpNBoxFhxvk4MLAQwvBFFhXKrVHJwPwKxXYldFak9sw8fU6OHADocjq1UqGAyiu7sbW7duTZnWpcZRqKA5HA6MjIygtbU1r5PO4w3hdP8lnOjxQaGXYU8li5v3WqFRyrMqGVHIGMTb+lLcIv17RyIR7Nu3L6eDWiTWl0yeWrKEQEtLCxwOB86dO4dgMAir1YqKiooVz+UFgo8+0o3eWR8iMQEapQwfeVM9PnldY87XWaJR4hPXNmLMFYJACOrKNFDLIQ561Ol04pBHk8kEmUyGUCgEl8uFoaEhxGIx0Yuz+zjoVHIxg6pXy+Hws9i18iOIbPjtJyUb15ueWZCpEJeKGk0W5BO4nJ6ezjkmkQxCCLoHLuJnpx0oKy2DUqnAyxOLYAnw/gPV4raYpuCTFf4eqDfikZGlSbwxXkC5QYUKQ/qZc8FgcM0Pm5FYTq6LaLoOgYsXLyIYDK5ICBgMBgiCgIWFBTQ3N4NlWYyMjCAcDsNkMsFms6GsrAxnJhbRN+dHOCaAARCOCfjxnyfwkTfXQ63IfYuqUsiwM86bol0MPM+Lh2rHJ9LUajVqampQV1cHnufh8Xhgt9uxMOHGWFCOLeWlMJQYEGJ5lGnTSwnLsmtykhSwiqJGM0kOhyNjIS4N0judTmg0GpSXl+e0BaM3JxwOo729vaDWKUIIhoaGMOGOQldSCnPJ0o2oLtPi/LQfH7hKAaWSWZZOT1b4u8WsxTXVClRUG6BRyrDVqkt5+CxFOknqyiPZb4DjOPT29kKv16O1tVUUC7rzoAMl29raRDunhbdUOIaGhtC/qAAIQbzMyhggzAp5iVo8hBAMDg6CYZgVY5PiQzDxSQcaC27cRvBU3xxG5j0YnZ+ERQ2orFF4vUJRTpwqlII6ClK1SfE8L57bmWxOezz0hmu1WnR0dMDpdGJgYAAsy4pTNOm0gmQIgoD+/n6o1Wo0NzcXVG9DX0ur1WLHtho8PzshboNjvACVXAYaC6Xb1FSFv7FYDFadAs3VJVnfZGlA5JVHoqdGEwINDQ2oqqpaFkoBgLGxMXg8HnR0dKzoUJDL5aLNE0JQteDBj871gNYEyBmgrkwDo6YwQaN2rtPpsHXr1qQ1n/GJtERxYwC8a18lfE02yGQy6JXA4uW6zKGhIbFH2mKxZD3pppgU3VOjbR/UbU1HYoZTq9Wirq4OdXV14DhOPNCBzq2y2WywWCyiJ0aTD+Xl5RnfKxM8z6Onpwdms3mpDUoQsKNCj4F5PxSMDAII3n+wNunNScw6cRyHwcFBWK3WjCUj8Uh1alce8b8Bj8eDCxcuYO/evSgtLV0WSqGdMzKZDG1tbRkXOoZhsKXSjH//YAe+8LsLsPui2G5R4+Otapw+fRomkwlWq1WMf2VLop1nIl3JiFEtE6/VYrHAZrOBEIJgMAin0ylOuvH5fGKRcjEHUKSiqKJGb2o2Z35mSggoFAqxz40QgsXFRTgcDoyOjopNvvPz89i2bVtOXQzJiMViohDTCQwKmQwfPdyAnmkf/BEODRYttlozC44gCOjr64PJZEJDQ0PSkpFUhb/SSVLrT74xNXrmBk0I0HgrwzBiy5PVakV9fX1O79FSY8SpT1297O/oNpXGq/V6vbjgpwvz0JH4NNueK+lKRuJDMHQgK51086c//Ql2ux33338/vvOd7+T8vrlScEM7XY2mp6cxPT2dVSU2bV0CsksIMAyDsrIylJWVYfv27bDb7RgYGIBarcb4+DgCgQBsNlvGk5qSEY1Gcf78eWzduhU2m23Zv6nkMhxoKMv6tegqaLVaRc8x2wpwmUwmbT+vQOh9s9vtSTsEgsEgent7sW3bthX2lS+J29RAIACHw4Hu7m4AgM1mg9VqXXYeB8uyOH/+PLZs2VKwE0DJVO9JF+/9+/ejsbFxTQQNKNBTo8WpAwMDiEajOHjwYFr3stCCWgCit3bw4EHodDrEYjE4nU6MjY0hGAyK2aNs3PJQKISenh7s3Lkzp9Pkk8HzvNjHmi6DmcoQIpEIXn311ZStYhIbD47jxDlnyRICbrcbQ0ND2Ldv36otVvFnXtDGdqfTuSybajQaMT4+jh07duR8jGU6HIEozk56EeMI9lUb0GjVr1i8CSE4e/YsZmZmiva+mchb1OiBJqFQCFVVVdi1a1dakSqGoNGSjf3794tBVqVSKZ7UJAhCUrfcarWuCMr6/X709fUVxeB4nsf58+fFiQrZQsWN53ncd999+Nu//Vts3555UqnE6pGtXYZCIXR3d2PLli2IRCLLOgQYhsHMzAxmZmbQ0dGxZqUMAMQyDJpNnZubw9DQEBQKBWZmZsCybMZtaja4gyz+4/QMBEKgkDHon/PjrvYqbC9fCp/QxfvChQv4yle+gt/85jfF+HhZkbeo+f1+9PT0QKVSZSwWLVTQsi3ZkMlkYtYl3i0/d+4cZDIZrFYrbDYbWJbF0NAQWlpaCg7McxwnxuNSHWCbjlgsho985CO45ppr8PnPf14a5X0FkJgQGB0dxfDwMCoqKmA0GnHp0iWEQiHs379/TQLjqQgGg5iamsKBAweg1+uz2qZmy5A9gBgvoKp0aVSYQh7DmQmvKGoAMDg4iA9/+MP49a9/jX379hXvg2UgL1Gjh6K0tLSI7ne6xxbSIZBvyUYyt9zhcKCvrw+BQADV1dWIxWIFpZtpgqGuri6vAXgcx+G+++5Da2urJGhXCDR2THcLHMfhwIED8Hg8mJ6eht1uh1arRVNT07pep8fjwdDQ0LLOmlTb1FAoBLPZnHM2lST8Id56R0ZGcO+99+KXv/zlmgoaADAZ5uKn/MdYLAZBEPDyyy/jqquuSvpF5JoQSPYexSrZAJamg0xPT6O5uRl+vx8OhwOLi4swGAxi9ijb5nwqaA0NDXkFXnmex6c+9SnU1dXh61//ejaCJilecUlq24SQpMM9aVE2bWWjvcp0oY5EIujp6UF1dTX0ej3sdjvcbnfaEMhqQePObW1tWW19aTbV6XTC4/FklU11B1k8/Ndp8IRALmMQZnkcba9CU3kJJiYm8J73vAc/+9nPcODAgWwuuai2XbConT59Gu3t7StuWPwMtHw8EDplI1lWMh8mJyfhdDrR2tq6bEtACIHP54PD4YDL5YJSqYTNZoPNZks5hZdmkhobG/O6NkEQ8JnPfAYmkwn/8i//kq3gS6JWXLIWNY7j0N3dLc4nS0wI0Pjszp07l51pGx8CcTqd4rTkdLZVKHThbmtry0tE46/Z5XIBSL1NdQZYnJtaBMsJ2FttwBaLDtPT07jnnnvw4x//GFdffXWqt0lkY4naa6+9Jk4fAIqTEMhlykYm6FY5GAxi3759GQUkHA7D4XCIzfq0qJAedMyyLM6dO4dt27ZlnIqaDEEQcP/990OpVOJ//+//nYsHK4lacclK1OITApWVlStCKXSAQnNzc8b4bKJtUYHLJ6aVjKmpKTgcDrS0tOQ1Hy0ZdJvqdDozblPn5uZw9OhRfP/738db3vKWXN5mY4gaHY3d3d2Nbdu2oaSkpGglG3TKRrb9n7xAMLsYASFAdZkaistfNu1vA5AxO5uMWCwGl8sFh8OBQCAAg8EAr9eLnTt35u2hfelLX0I0GsUPf/jDXLfkkqgVl5S2HY1GAbx+Cvu+ffvEMwSA10uZqPff0tKSs1fEsiycTiccDgfC4bC4eKZrCUz5QS73Wft8PjQ3N69a72WybSqtl3O73bj77rvx7W9/G9dff32uL72xRK2vrw91dXUwGo0FJQSApZVmYWEBra2tWRtJJMbj+38cxbB9aa7bNpsen37bVmgUDPr6+qDT6bBt27aCV8JQKITXXnsNBoMBoVAIOp1OdMuzSY8TQvDVr34VLpcLP/3pT/PJikmiVlzSihpNCNBtXPxCTUfEE0Kwe/fugkWE53lx8fT5fCgtLUV5eTnMZnPG16bTQGKxWFGuJVvit6lPPPEEvve97+Guu+7Cgw8+mM/LFdW2C/ZRaVdBIQmBQqZsPHXBjoH5AKpKlwKiFxeCeLxnDtvlDlgslmVjvPOFxvf27duHsrIysb8tMT1us9mg0+lWCCghBN/85jcxPz+Phx56aF3T/BLpod59JBLBgQMHViQEYrEYent7xd7JYmwb5XK52BIoCAIWFxfFs3P1ej3Ky8thsVhWLPSEEFy4cAEKhQJ79uxZ0+w5rS5gWRZdXV34l3/5l6Ik84pBQcW3wJKIud1ulJSU5LWPp72SWq02rykb094ItCq5+DyNksGrg+N4y9u35VU3lkg4HEZ3dzd2794txvfo8WQlJSVobGwEy7JwOByiMMfPxGIYBg888AAuXbqERx55RBK0KwB6/2g5Eh0nFQqF0Nvbiy1btiQd6lgMZLKlY+1MJpPoDdntdkxMTCxLYqlUKvT19Yk2uB7lQF6vF3fffTe++MUv4siRI2v+/qkoaPvJsixCoRCmp6fhdruh0+lQXl4Oq9WalcDRBtuKioq8Vf7JvgX89rVZVJWqwfM8Ls24cUd7Ff7m8I6kj7/kCOIbTw7D7mdxeJsZn7thG1SK5N4lDRLv3bs365PmeZ6H2+2G0+nE9PQ0vvWtb0EQBDz33HOFNqtL28/iknb7yXHcslCK1+vFwMBAUZJX+UITDQsLCwgEAjCZTNi+ffu6THbx+Xw4evQoPv3pT+Puu+8u9OU2xvbzO9/5DqxWK2666Sbs2LFj2aoyPj4OtVqNiooK2Gy2pPEx6gEV2uj79t02jLvDOD3mgt8fwFVNNtxzdfLCR4c/ivf9/CyCUR4EwOxiBI5AFN85urI4MBgMilvOXNqo5HK5GGt74YUXYDAYcNVVVxUtGyWxurjdbnz1q19FZ2eneDjQ3Nwcpqam0N7evmqlGNmg1WpRVVWFhYUFNDU1QS6X4+LFi4hEIrBYLOL5uavttQUCAbz73e/GJz7xiWIIWtHJ21Pr6enBb3/7W5w6dQpVVVXo7OzELbfcIjaG08kFDodj2RghlUqFxcVFcURRMVY9n8+HV871YdeuXagrN6W8qSfOz+EbTw4jHHt9qJ9cxuD8F69bdhJOIBBAb28vmpub8/KuCCH4+c9/jlOnTuHRRx8t1g9B8tSKS1LbZlkWp06dQldXF1577TXYbDbcdNNN+OhHP7qugga8fvh3Yn0kTTTY7Xb4/X6UlZXBZrNllWjIlVAohHvuuQcf/OAHce+99xbrZTdG9lN8wOVg5fHjx/HEE0/AZDKhs7MTt956q1jHFQ6HsbCwAIfDAY7jEIvF0NLSgrKysoI/AG0HaWlpyXjQysmeefyvJ4YQihM1hYxB95euE4WQFlIW0hf6y1/+El1dXTh58mQxT4aSRK24ZLTt5557Dj/4wQ9gNBpx9uxZXHPNNThy5AgOHz68Zt0BFLqz2bFjx7IC30ToubIOh0PsaMglJJTpGt773vfi6NGj+NjHPlbQayWwsURt2YMvp5ePHz+Oxx57DFqtFrfffjtuv/12VFRUiIe12mw2uN1uCIIAm82GioqKvH78tB2ktbU1q1U0EOVw2w//ClcwBk4g0CpleO/BGvz9DUvbVVr0W4ig/frXv8bDDz+Mxx9/vNixDknUiktOth2LxfDCCy/g+PHjeOmll3DgwAF0dnbirW9966pP4aA7h1xiuwCWhYScTieUSiXKy8ths9lyvuZoNIr3v//9uPnmm/GpT32q2FvcjStqy554uSCwq6sLJ06cgNfrxeHDh/G5z30OtbW1YnW+3W6H3W4Hx3HiUWHZiEG+7SDuIIufvDiBeV8U1zaZcWdbFRiGweLiIgYGBnIq+k2kq6sL//f//l88/vjjqzE/SxK14pK3bXMchxdffBHHjx/HCy+8gNbWVnR2duLtb3970c9spXaZTcdCJsLhsBgSIoQs62hIB8uy+NCHPoTrrrsOn/3sZ1cjZndliFo84+Pj+N73vof6+nr8/ve/RywWw6233orOzk6x1oeeI2q32xGJRESBSzbNNlUfZ754vV4MDg4WJGgnT57ED37wAzz++ONF2VYnQRK14lIU2+Z5Hi+//DK6urrw7LPPYvfu3ejs7MQ73vGOgkXI7XZjeHi4ILtMBS1DcjgcaRMNsVgMH/7wh3Hw4MHVnCRz5YnashckBAsLCzhx4gROnDgBv9+PW265BZ2dnWhqagLDMOA4Dk6nE3a7HaFQSPzCDQYDRkdHEQqFsurjzAYak2tra8s7EPzkk0/iX//1X/HEE0+kjXcUiCRqxaXoti0IAs6ePYtjx47hqaeewrZt23D77bfjpptuytlzp1UEbW1tBQ90zESyREN5eTn0ej0+9alPYdeuXfjHf/zH1cyqXtmilojD4cDvfvc7nDhxAk6nEzfddBNuv/127N69GwzDiF/4wsICXC4X1Gq1OH670C/Z5XLh0qVLWY9oScazzz6Lr3/96zh16lReDe45IIlacVlV2xYEAT09PTh27Bj+8Ic/LKsQyOTJz87OYnZ2Nqd2wWJBEw0zMzN497vfDb1ej6eeemq1D9neXKIWj8fjwcmTJ9HV1YXp6Wm84x3vwJEjR9DU1ITBwUEYjUaUlpaKc9DoipLrMWEAxAF57e3tea+Ef/rTn/CVr3wFTzzxxKpVmMchiVpxWTPbjq8QePzxx2E2m3HkyBHccsstKxbCyclJuFwutLS0rFv3CR2NVVpaive+973Ys2dPxsqCAtm8ohaPz+fD448/jt/85jc4c+YMbrvtNvzN3/wN2tvbxRN7vF4v7HY7PB4PjEaj2COXSeAcDgfGxsYKcu1feuklfP7zn8fjjz+e13FjeSCJWnFZF9uOrxA4efIkdDqdWAI1OTkJrVZbtNBKPhQwGqsQ3hiiRhkZGcGf//xn6PV6HD9+HAMDA3jrW9+KI0eOiKdX0XNBFxYWxD5UWpuTuNrRWEWywZbZ8te//hV/93d/h5MnT65lE68kasVl3W2bEIKxsTEcO3YMP/jBD1BXV4c777wTR44cQVVV1Zr3cxY4GqsQ3liilkg4HMbTTz+N48eP4/z58zh8+DCOHDmCa665BgqFAoQQ+P1+sTZHq9WKtTlOpxNTU1N5TwUFgLNnz+Jv//Zv8fvf/x5btmwp7odLjyRqxWXD2LYgCDh27BiuueYanDhxAo8++ih4nsett96KI0eOoK6ubtUFrgijsQrhjS1q8USjUTz33HM4fvw4Tp8+vaLim44IWlhYwNzcHDiOw7Zt21BRUZHXtrO7uxv33XcfTpw4sR4Ha0iiVlw2rG0TQjA/Py8KXCAQECsEijEbMNn7ffOb38TExMR6jcaSRC0ZtOK7q6sLL7744rKK79nZWbjdbuzYsUOcNho/wyqbzGd/fz8+8pGP4NixY9i5c+cafKIVSKJWXK4Y23Y4HHj00Udx4sQJuFwu3Hzzzbj99tvzmuacCCEEDzzwAC5cuIBHHnlkvQYvSKKWifiK78ceeww1NTX4zGc+s6ziO766GoAocMlq1QYHB3HvvffiP//zP7F37941/SxxSKJWXK5I23a73WKFwMzMDN75znfijjvuwJ49e3KOgRFC8OCDD+L06dP4zW9+s+blI3FIopYthBB84xvfwJve9CY89thjeO6557Br164VFd/RaFRs1+J5XhQ4nU6Hixcv4gMf+AAefvhhtLa2rufHkUStuFzRtg0stVA9/vjj6OrqwujoKG688UZ0dnaira0tqzHgP/nJT/DHP/4RXV1dq17gmwFJ1PIlvuL76aefxtatW1dUfNP2Ebvdjp/97Gd44YUX8MADD+COO+5Y56uXRK3IbCrbDgQCOHXqFI4fP47BwUFcf/314ky4RIGjo7GeeOIJ/O53v1v3kUqQRK040Irv48eP48knn1xR8T01NYX3ve99uP3229HW1obOzs71vmRJ1IrLprXtcDiMp556Cl1dXTh//jze8pa3iBUCcrkcv/zlL5dN0tkASKJWbBIrvvV6PaampvCLX/wChw8fXu/Lo0iiVlzeELYdjUbx7LPP4vjx4zhz5gzKy8sRjUbx9NNPr8sY8BRIoraaEEJw7tw59Pb24kMf+tB6X048kqgVlzecbcdiMfzoRz/CnXfeudq9nLkiidobFEnUiotk2xuHotr2+jSYSUhISKwSkqhJSEhsKiRRk5CQ2FRIoiYhIbGpyNToJQWnJTYrkm1vUiRPTUJCYlMhiZqEhMSmQhI1CQmJTYUkahISEpsKSdQkJCQ2FZKoSUhIbCreUKLGMMwtDMO8yDCMl2GYeYZhfsowTG5HZ0tIbEAYhnkbwzC9l23bxTDMowzD1Kz3da0HbyhRA1AK4OsAqgHsBlAL4NvrekUSEsXhAoB3EkLKsGTfFwH8aF2vaJ244kWNYZg6hmFOMAzjuLxC/SDVYwkhvyKE/IEQEiKEeAD8FMCb1+5qJSSyJ0fbXiCEzMb9FQ9gzY882wisy9ExxYJhGDmAxwE8D+ADWLqRB3J4iWsB9K/CpUlIFEQ+ts0wTD2AHgDGy4//6Cpf5oYk0zy1DQ3DMNcAOAmgihDC5fjcGwH8FsBVhJDh1bg+CYl8KdC2zVgStD8RQl5ZjevbyFzp2886ABN53PSrAfwKwFFJ0CQ2KHnZNgAQQtwAfgHg9wzDXNG7sXy40kVtCkB9LjeOYZh2LK2AHyaEPLdqVyYhURg523YCCgDlWNqKvqG40kXtNIA5AN9iGEbPMIyGYZiUgX+GYfYB+AOA/5cQ8thaXaSERB7katt3Mgyzk2EYGcMwNgDfAXDustf2huKKFjVCCA/gNixleSYBTAN4d5qnfA6ADcDPGIYJXP5PShRIbDjysO0aLC3YfgC9AAQA635Y7XpwRScKJCQkJBK5oj01CQkJiUQ2nagxDPPjuK1l/H8/Xu9rk5AoBMm2s0PafkpISGwqMqWLJcXbOEgz9YuLZNsbB+kwYwkJCYlUSKImISGxqZBETUJCYlMhiZqEhMSmQhI1CQmJTYUkahISEpuKK3IsCSEE0WgUsVgMSqUSCoUCcrkcDCNVPUhc2fA8j3A4DJlMBoVCAYVCAYZhJNvOgUzFtxuulofnecRiMcRiMXDc0qgpesMVCgWUSiXkcvlmFLlN9WE2ABvKtgkh4DgOHMeBZVkQQkAIgd1uR1lZGQwGg2jbm1DkivphrhhPLf6mMwwDmUwm/kf/ned5UeiA10VOoVBAJpNtNkOQ2CQIgoBYLAZBEETBona9uLiIkpISsCwLlmUBQPTiqMjRx0oscUWIWrKbnihQiX+XKHI+nw8GgwE6nU4SOYkNAbXRWCwGACvsN96m5XK5+PcAJJFLw4YWtWQ3PVshSnzs/Pz8MiFjGEaMWUgiJ7HWEEIQi8XA8/wKW2UYRhSvZIs3IIlcOjasqCVuNxmGgTcUQ5QTYNIpIc9Rf+hrxBsDjc3Rf5dETmIt4DgOoVAISqUyq4U6Xdw7mcgRQlaIXHxCbbOL3IYUNUEQMDU1hZKSEuj1ejAMgz8OOfDCsBMMw6BMq8D7DtbAqMpfdOIFDkgucomGIImcRCHQnYfb7cbExARaWloyPidXm0sWhqHVAtFoFMCSAFIvjmZXNxMbSrKpdxaNRuHxeBCNRsEwDCZcITw/5ECFUYPqUg2CLI/fd88X9b2pyMVnTlmWRTAYRE9PD3w+H0KhEFiWhSAIaVdPCYlEqPcUi8Ugk8lysp9CbI0mHahdy2QyCIKASCSCYDCIxcVF+P1+RCIRcBy3Kex6w3hq9KbTZAD98gHAG45BxjBQyJZWFLNOhbnFSM6vnwvxnpzH4xFFjq521KXfxGl2iSIhCIJYpkHtKlt7jI+vFYNknpwgCOju7kZtbS30er3oyV2p9Z8bQtQSbzoVNXozzTolBAJwvACFXAZXkEWZksfMzAzMZjN0Ol1WX3whNycxHgdIwVmJ9CQrQwKWbIku2OsN/b0RQlZ4cpQrTeTWVdRS3XRg+QpVZ9bhXXvK8fSAfekfIz601MoAaDE6OopwOIySkhKYTCaYTCZoNJpVvW4pAyWRiWRlSJRcvK9ie2qpSFUuRT25cDi8zO43ssitm6glbjcTv5j47ScAvLnJgp1WFc719KFxTwW2NNSD4zjU1dWBEAK/3w+Px4PBwUHEYjEYjUZR5JRK5ap+lnQi193djV27dolBWUnkNjfZlCEl2vZGgBCS1CYTi4GvBJFbF1HjOC5j7VniCmW323Hx4kUcaN4Dk8kEnueXPdZoNMJoNKKhoQGCIGBxcREejwdTU1OioS0uLkKn0y3Leq4G8Tc7FAqJMZQ3cpr9jQAhBD6fD36/H1arNeWPeyN6ajT0k831ZBK5+NKo9RC5NRW1ZLVnqaBxB0EQcPHiRQQCARw8eBAqlSrj+8hkMtFLA5ZEtLe3F4uLi5ibm4NcLofJZILZbIbBYEgrKMUwqGzT7PHGsN6rnURu0O1mIBCA0+mE1WrFK2NuDC8EUGXU4LodVqgUr8fUNlqWke6YciWZyK13u+KaiZogCIhGozh79iwOHDiQ8YPJZDKwLItXX30VFosFHR0dK+IS2aJQKKDRaFBbWwuDwQCWZeHxeDA7Owu/3w+1Wg2z2QyTySTWxRWLVF5osrgFFblbbrkFL774oiRsVwDx201awE0IwX+emcHx12bAMIBACP4y6saXb94JuYzJafu5lp5aMXYL6doVX375ZTz//PP49re/XfD7pGPVRS3+ptMC12x+rMFgEPPz82htbYXZbC7qNalUKlRUVKCiogIAEA6H4Xa7MT4+jmAwCL1eL4qcWq1edXGJNwRCCILB4Kq+n0RxSNbqJJPJEGI5nDg/C3OJCgrZkij1zvpw0R7ArkpDRqFaD08u2+1nJvwRDj/7yxQuzPtRZdTg42+pR1XpUuLO7/evSZhlVUUt8aZn84EIIbh06RJcLhfq6uqKLmjJ0Gq1qKmpQU1NjSgqbrcbw8PDiEQiYFkWCwsLMJlMWW1/C0Wqedv4JCtDApZ2GDGOXyqRuHwLGYaBnAFYXhD/nC0bLaaW6TW+9fQlDMwHoFcr0Dfnx5ceG8KDd++FXq1AKBRCSUlJka44Nasmaqluejqi0Sh6enpgMpnQ2NgoBtXXEoZhUFJSgpKSEtTX1yMajaK7uxuhUAgzMzPgeR5lZWUwmUwoKyuDQpH+K8zVIDdaVkxiOenKkIAlUVPLCFpqStE9vQijVoEQy6NUq8RWq36drjozxRA1X4TD4EIQJt1ST6taIcNiOIYRZwgtNUZxF7TaFF3UMt30VLhcLgwODmLXrl2wWCyYn5/fED9whmGgUqnQ2NiIxsZG8DwPr9cLj8eD8fFxMAwjJiVKS0uXfd58gq/hcBg6na7YH0OiCGQqQwIgFo3//Y1NeOjlSVyY86PJVoL/580NKFFn/rkJgoCJiQkoFApYLJa8PDXP5cEPFr0SSnn2271CRW3pvQgEAsiZy7FiAqgvJ0gCgcCa7LyKKmrZ3PRkzxkZGYHH48H+/fvFwtmNkiFKXMHkcjksFgssFgsAIBaLwePxiCUnKpVKzKxqNJqcy0cCgcCarGYSucFxHEZGRlBXV5c2g0eTAHq1Ap9669ac3iMSiaCnpwelpaUghGBgYACBQEDM0GfaGRBC8MygE38d94JhAJNOifcdqIFJt7p1mhSdSo47WqvQdX5u6XoAdNQa0WRbsudgMIj6+vpVv46iiRr1znLdbvb29qK0tBT79+9f5uVkyhAtLCzA6XTCYrFktQ3Ml0xuuVKpRHl5OcrLywEsGabH48Hk5CT8fj9isRhmZmZgMpmg1Wozfi9rFXeQyI74ncf8/Dzq6+vT3sN8CmvdQRZdZ8YwNjWHmzq2YuvWKgiCgIaGBoyNjUEQBHi9XoyPj0Mmk8FsNovlSPHXMuoM4eVRD6pLNZDJGDgCUZzqt+P9B2vy/vy58v6D1Wiy6XDJEUSlUYPrtpshv9yzfcXE1PLZbhJC4PF4MDAwgB07dsBms614TCpPLb5uraqqKqubXejny+W1NBoNqqqqUFVVhXA4jIGBAQiCgEuXLiEcDsNgMCzLrCYieWobh8RWJ7lcDp7n09p4rqLmDrL4+MNn4PRHoNdq0fenWfydXIVrGpdqLOVyObRaLSorKwEsdam43W5MT0/D7/dDp9OJdu+LcGAYQHZZRMq0Ssz7ogV8A7nDMAyubjTh6svXH88VEVMTBAHBYBBDQ0PYt29f1hXJIyMjcLvdy7abyR6XaBw0kWA2m9HW1oZYLAar1Qpg5c2mZRl0G5gvhQRQCSFQqVSoq6tDXV0dBEFAIBCA2+3GhQsXwHEcSktLxaSDUqlcsxsvkZrEVicqYjKZDDzPp227yyVswnEcfvX8a3AHWdSXl4EBgxDL4ZEzs6KoJaJSqVBZWYnKykoQQhAKhcRM/ZQ7DI9XBoOcQ4leB3eIQ5N148Rng8HgxvXUEm96KBTK6ofPsixCoRBisRgOHDiQccWLNw6v14v+/n7Rs0sMwifebFqWMTg4KMb5SkpKoNVqc9qqFiJqgiCs2FLTdq4tW7aIrVt0uwoATz31FLxeLyKRyKo35kusJN2YbeqppSNbW+F5HqdPn4ZWXwq1hoC5fKCSXMYgxr2+mKcTSYZhoNfrodfrUVdXh2ZBgLJnCs8POcDavTCpGexqsMHr9cJoNK57K14gENiYopZ405VKZcYbDSzNJLtw4QK0Wi22bt2a8QumN5MQgunpaczMzKC9vT2rzGBiWQbP8+jt7YXf78fCwkJOW9ViiloicrlcvA5gKenw9NNPY2xsDE8//TRuv/32vN5XIj8ylSHJ5fKiZOTn5+cRDodx9dVXo45T4NTFXiyGY1DKZQiyHN6zvyqv15XJZLiprQHX7q4ByxNo5QS+RS/m5+cxPDwMjUYj2ls28d1iEwqFYDAYVv19chY1juOSrmKpIIRgfHwcdrsdHR0dGBwczMowqKvf19cHADh48GDejehyuVzcBtI2qWRbVRrMT7z+fG9+pvhLIkqlEvX19Th69KgkaGsMXawBpLxn2Xhq6aDxYBpiMBqNMAL4Ruce/PKVSfgjHN66oxq37rMBcQev5FoFoFcrQAMYmstJLEKI2DkzMjKyIr67FmzY7Wf8WZuZiMVi6O3thVarxcGDB8XnZmMY0WgUDocD27dvR11dXVFWFfoa6eIS0WgUZWVlMJvNKCsrW1VPLRlrdeMllkOTXOkW3EJELT4e3N7ejpdfflm0rV2VBvzzkb3iY+e9IfTNLEIgBCU8uzTCPsrh+WEXIjEBb95mQnVpbqEJhmGg0+mg0+lQW1sLQRDg9/vFxT0YDGJkZESM767GVnXDJgqy/YEvLi6ir68PTU1NYo8lkF12yOl0YnBwEAaDIW1dSzGqoBPjEjR9TotraeyOnhuay/vFi1qI5fHa1CIEgaC11ohSbfJgcygUEstDJDYW2S7IidDQy86dO8XEFvXAEu3JFWDx7IADaiUDOcNgYCGCfTHg6yfOwhOKgRBA9hyDn/9NC/ZW5b+Vk8lkKC0tRWlpKRobG3H69GkYjUY4nU5cunQJKpVK3KoWa8iDIAirPtsQWKWOgomJCczPzyeNgaUTNUIIRkdH4Xa70dzcjLGxsbTvtRoxgfh4GwA4HA7MzMxgZmYGPp9vWQo9cauaiCAIkMvlWAzH8N+P9cPuX2r7MmoU+N7de1BpXLnaSp7a+pHJnnL11AghmJycxNzcHDo6OpbZC/0dJHpEk54QlAoGRo0CDBjoVAx+2+uBPcCC41/fhn7tyYv49Yc7sr6WTNfJMAxsNptYXhWJRJYNeTAYDGJRebJSpGzeY62K6YsiarT8gsbA1Go1Dh06lNSFTSVqdKuq0+mwf/9+RKPRtB5dMQSNEAK7n0U4xqNErYC1ZGWzukKhgF6vx/bt25NuVUtLS8W4RGJWlRrtb87OYd4XhVGz9O/eUAw/fWkKX7lp+4r3k0Rt45KtqDEMg1gshoGBAchksqTx4FSxMgXDgBde/3siMPCxwjJBAwB3MJbnp1hJsnY+jUaD6upqVFdXi5Ol40uR4kM0ucS61yI5UZTtp1wuh8fjwdDQELZu3SoWCiYjmWH4/X709vaisbERVVVLmZ9sjhErdPvZM+vHhTk/ZAwDgRB01JViZ8VyQYl/j2Rb1cXFRfEcR4ZhRC/OaDSKomb3hyCLu0ylgoHdn7wocq3S3hK5k232kxCCM2fOoL6+HrW1tUkfk2pxb7TqMTjvg90XhUwmA0cI3rJFjwuOKCKxpcerFQzetLV4wf1Ms9QYhllRiuT1euF2uzE6OgqFQiHafUlJybpPmClKRwHLshgaGkJra2vGQGDizZybm8PY2Biam5uXpXvTZX0EQcDAwACcTif0er3Yi5lLXVcgymFgzo8Kgxqyy6tj94wPjRadOKGUfr50fX7xE3bjh08ODg6KpSX7Ko348wjACwQMA8R4ggP1ZUlfMxQKScW360Q220+aIU2F3W5HMBhER0eH2B+c6r2S2XeJRoF37LZh1OEHw8ggN/PQyQl4ZQn+7cUJ8ALB4W1mfOEd27L7UFmQq3OQ2P8cjUbhdrsxOTkpdsQkFr6zLLsmY7uAAkWN4zj09fWB5/msBA14XdQEQcDw8DBCoRAOHjy4IoCYaiVjWRbnz5+H1WrFVVddJaap6YErZWVlsFgsKC0tTesWczyBTMZAxrxe9EgIwAkE8V99Ljc8fvgknQsXi8XQwDhxtZXDi/MxMIwc79xjw/sOVid9DRq/kNh4pNt+EkJw8eJF+P1+lJWV5by4x6NXK7C7cqmJfW4uAI7jcN/henzszXUggGizxaLQHY9arRZbA5MVvpeVlcHtdq/ZYp339pNuGbds2ZLT4awymQzRaFQc093e3p5y5HXia/p8PvT29mLHjh2wWq1gWXbZdpC6xTSDo1arYbFYko470avlKFEp4A6xMKgVWAzHYNEroVEud8PzKcug169QKFBWVgabzYbWFgEerxculxu+RS96zp9bVgBM30OKqW1cUmU/WZYVp2t0dHTg/PnzGbep2dafxT+OYWjfQXEp1tRbIHnh++LiIk6ePIn+/n584xvfwJe+9KWivFcq8uoomJ6extTUFFpaWlBSUgKv15t1VigajWJ2dhbNzc1iejsZiSvZ/Pw8RkdH0dbWBr1en9QgEt3icDgMl8uFS5cuwev1QhAEVFZWLh2bJ5fj2u1mvDa1CE8ohhqTFm21xhWrYLGKb2UyGSxmMyyXBTZ+q0qzqiqVCizLSqK2TuST/Uxs3wOyK1vaKKO1gPwX7mygXTO33HILBgcH8YUvfGFV3ieevEQtGo3i0KFD4vZOLpcvOz0m1fOmpqawsLCA2tratIIGvG5gdBvn8/mSblPTodVqUVtbi9raWvT398NkMsHr9WJsbEwcwre/ygKdzpzSoFer+DZxqxoKhXD27FlMTk6is7MTTz/9dF7vKbF6xCcK6OI+PT29onSpmKK2FuJXTE8tFTRWvNrHUwJ5dhQ0NTUt+6IVCkVaT43nefT394NhGGzbtk08Fi4ThBCcO3cOer1+xWlSuUIzONXVS7EsWodDT3g3Go2wWCwrSjMKFbVsbiLNql577bWwWCx48skn83o/idWFemo8z+PChQsAsGxxp2QjatmOKFqLTOJaiNpajtQqSp1augBqKBRCd3e36DHZ7XaEw+GMrxkKhRAMBtHY2CgKUTGJr8MRBAE+nw8ulwsTExNiAa7FYsn7PEQgd7eeLhTrPU1BIjlyuRwsy+L06dOiPaca6Z2vpyYIgtibabVawfP8qntqhdh4tqxlqVJeopZ4Q1KJmsPhwPDwMPbu3YuysjIAK4Otr0548ej5WTAMcFd7NdrryuB0OjE0NAStVrsqgpYIHZVMr5FlWbhcLkxOTsLj8UCj0UCpVMJsNue0/S0kySCx9mT63t1uN7xeLw4dOoTS0tKUj8vXU4vFYuju7kZpaSmqqqrgdrtht9vFOrJitizFU6wzP9OxlhOdi+apxW8p49udDhw4sKytIj4u8eqEF/948oI4qfO1CS/++9VmmIRFHDhwAGfPnk37vjQQX+wbolKpxBT11NSUOAduenoaAEQvLlMvaK6ilutUD4m1gcZ1vV4v9Hp9WkED8vPUgsEguru7sXXrVlgsFrAsi9LSUhiNRni9XigUCoyNjSEcDqO0tFQMlRQjRrUW28+1HH5a9O0nbXfS6/Urzh0Alt/wk91zkMkYGDVKAAQLngCeHnLjX993ddofNx1SCUCMcdDXzuUEq2zR6XSoqqpCY2MjYrHYsrFFJSUlYtlIYnFhrqImFd5uPFiWRW9vLwwGA/bv349XXnkl43OyFTX6GHqSWnNzM/R6vZh0iz/5vbKyUgyVLC4uwuVyYWxsDEqlUsz45zsjba1iams1qCHv7Wc8VNRo7Vq6Vqn4G84wS2Oj6FhwmVwOq8WSUdDoTVer1RAEQRQ5+rrxhycXKnCJN1ypVC7LWgYCAbhcLvT19UEQhKRtUtkSCASk4/HWkUS7ppNmtm/fntMPMtvtZ3zD+/79+6FUKsXkkkKhgMfjwfT0NHbu3Lls8TYajeJxjJFIRCxbikQiYvF5Lj2ZaxFTW8v6y6J5ajTQTmvXUhEfU7urvRqnx1yY94SgVqmgVilwR1vqGBoVrvgBlVQ06A2kjfVU6Oj/5zslIN0qxjAMDAYDDAYDtmzZAo7j4Ha7xUmjkUgECwsLsFqtWU02kE6S2jjQWkxaF5kL2Ta+T05OQqVSYf/+/aLnRm17bm4OU1NTaG9vh0ajERdvat/A0u9BoVCgqqoKNTU14tgsl8uF0dFRqFSqZV5cKqSYWgKCIGBychLBYBCHDx/OOP8/PqZmYQL44A5ghKuEUqlEZ2sVdlWubBGiNzOb80TjvTNqCIFAAH6/HzKZDCzLio/J9uSrbFcxhUIhHpdHCMErr7wipv85joPJZBJbuJK9t9TMvv4IgoALFy5AEISk5RqpWFo0l05yyjR3LRaLYW5uDkajEfv27Vtm2wAwMjICn8+Hjo4O8fcUv3hTjy7Z4l1aWioOeaTF58PDw2K7EvXi4u1PiqnhdTc9Go2iu7tbdIezOdCE3vDBwUGEw2HcfcPVGQ9ozVbQkr3X4uIiLly4gObmZpSUlIjeXraxuHxvOD1Srb6+HvX19eA4Dl6vVzz0WKPRiLE4uopK28/1hRCCs2fPoqKiIqdpy6f65vGj/xpHNMbjmq1mfLDFAFmK7SdNCFCBibdBugCqVCq0tbXlvHgnhmCUSiWqq6tRW1u7ooWQ2p/FYlkzUdvwnlr8NM+SkhL09/dn9Tye5+Hz+WA2m7Fz586MXyaNn+UqaMDSxISxsTG0tbWJwpHJEBIFrlg3XKFQwGq1wmq1ivPiE1dRKnYS6wPDMGhvb8+4OMfbRM/0Ir73/Ch0Sjm0WiVeHnVDiEVwb6txxfNoqVJzczNcLhfcbjfKysqg0WjE/lEqqLmQKgSTuHiXlZXBZDKBYRjR/gYHBxEKhaBWq1FWVpZyF1EoG17U7HY7hoeHxWme9DDjTAQCAXR3d0OlUmHbtvSjUwgh0Gq16O7uRnl5edZxKcrU1JR42Euy2rJ4Q4gXNeoVUkMoJIiaLhZH58XHN+L/9a9/xR//+Ee8/e1vx6233prXe0oUhkqlynhOQXynSN+sD7xAxHFVJWoFeuZCEJqX/4ATEwIKhQKCIKC/vx8sy4JlWTQ2Nqacv5YLmbw4QgjUajVqampQV1eH2dlZLC4uirsIrVYrenH5TLlNxlpOn8lL1CwWy7JpntkERu12Oy5duoSWlhb09vamfSy9Afv27UM4HIbD4UBvby8EQYDVaoXNZks5jI6OgIlGo2hvb89q1YlvOgcgChvLsvB6vbBYLIjFYjmXjGSbmKCN+IcPH0Z7eztuvvnmrJ4nsfZQW6e2b9KrLmfxl7y3KCfAplOKAkJn//E8jwMHDgBYsm+VSoUtW7bAYDBgeHgYDQ0N8Pl8eOWVV1BaWgqr1QqLxVJwHVo6Ly7+//V6vdghEQqF4HK5cOHCBfA8L9ZlGo3GvBf4De+pKRSKZT/YdB80sRA33aC4xPiZQAC1VostW7Zgy5YtiMVicDqdGBsbQzAYRFlZGaxWK8xms2hs/f390Ol0WZ8Ynwwa3+jv70dtbS1MJtOKrFOykpHZxQieHXQiyPJoqzGAyTHbGggElo0hkth4JC7g1++w4sm+BQwtBMBgaarxx99cDUHwg2VZdHd3w2q1oqGhQfSa6OI4NTWF+fl57N+/X/xdEEKwuLgIh8MhZjBtNhusVmvGMzGyIdGLi0QimJ+fF7P3DMNAo9GgtrZWjAW73W5x8Gm6usx0hEKhNYsXF/3glXjoEEm1Wp20EDee+HQ1wzD41ZkZnDg/B4EQvH2nDfe9pQFKpVKs9Kfpa6fTiZGREahUKoTDYVRXV6OxsbGg645EImJ1Nx0nkynr5I3w+PGfJ8CAgUohQ9f5eezSCLg6h/cNhUJr0hYmkZpMC2FiZlOtlONf79qHVyc8CMV4NFcboRYiGB934tVXX0VTUxNsNptoJ/T1h4aGwLIsOjo6lnljDMOILXvbt28XdyoDAwOIxWKwWCywWq0oLS0tONZL43jbt2+HyWRa5lTE/xatVquY0Y+vyySEZN1dQwjJKpFYDFbtXcLhMM6fP4+6urqMcQIqEHQVe+GiC8dem4VJp4SMAZ4ddMCsV+J9B19/nfhTn0KhEM6fPw+TyQS32w2HwwGLxQKbzZbzsXahUAg9PT3YuXNn0kNeU8UrLtoXEWYF1JSpwYCBnCEYcOTmqUkDIjc+yUItKoUMb9r2+ujuyUk7XC4XDh06BJ1OJ3pAMpkMHMeht7cXRqMRO3bsyGibWq12WQbd7XZjZmYGAwMDMBgMsNlssFgsOQtGMBhEb28vdu3aJfY8088HpE426HQ66PV6cecU311jMBhELy4+jr2WJ0kBReooSMTtdmNgYGBZI3sq4gWNZjh7pn1QyBgoLveE6lRynJ9aXCZqFFqysW/fPhiNSxmnWCwmTtwIBAIoLS2FzWYTt6n0fYcWggixHHZWlkCvUsDv96Ovrw/79u3LKqgZH6/QaTRLgsfIQEAQibJQyBiwLAu5XJ5VLG4ta3kk8iPd4Su0Q2B2dhYmkwk6nW5ZKVIkEkFPTw/q6+vTHk6UisQ6SJ/PB4fDgfHxcSgUCnGbmmmbl42dJ1u8k4VgbDab2F3j9/vhcrkwPT0tHkJksVhEm16rQQ1F9dToIMjZ2Vns378/Y3lCfIdA/A/eZlCBizsmLMoJKDeszMIkK9kAllqZ6AnstFeOblPVajXMFiu+/l9OvDblg1y2tF380Z1NCC6Mo7W1Na+9/+7KElhLVJjzsYDAweH24mNv3b7sR8BxXNrCX8lTW3/ymX4LvJ4QoAmuwcHBZSVCdPHdvXt3xoU+2+ukhxE3NTUhEomIJSPRaBRmsxk2m21FiQY99S0XO89U+EsrH/R6PUpKStDY2AiWZcXDWJ544gn4/X4888wzuPHGGwv+7JkomqjJZDJxn53snMNEOI5b5p3Fc1tzJV4e9WB2MQJg6fDfD1y1vHZncnISDocjZclG/HXRE5+2b9+OUCiEh/8yijPjXrCXF1wmyuOLJwdx7L6r864T06sV+NR1jXjhwgwujU/ig+/ai331r0/3TeXOx2dUJVHb+CQTtcSEAMdxUKlU+Otf/4qysjIolUq4XK4Vi28xocF9WmjrdrsxNzcnBvetVisYhsHExATa2toKqofMVDJCF+/y8nJUVlaioaEBv//977MeDlsoRdl+RqNRBINBWK1WbNu2LWPAkGEY2O32lLEAg0aBb9+5B70zPvAE2FNVcnmSR34lG/HodDp4eLUoaABAACyECEZGRsQYRT6pdCEahI2dw9vftT/pyfSJJSP0P/qduN1uSdQ2OImJgkAgIAbb6VBHhmGwd+9eCIKAoaEhzM/PQy6XY3h4WNwiruZxcXK5XDxtnW4Lx8bG4HK5UFJSgvn5edhsNuh0uoK3hNkU/tJ421rVXubtqdF5UPSEJ71ej+rq6oyCxvM8du3aJZ73qdVqxRsQ73FplHIc3LI8UF+sko09VQZolTKELx8OK2eAPdVlqKmpgcPhEEe65JJK93q9GBwcRGtra8bHJ1vpzp8/j76+vrw+j0TxyGX76XA4cPHiRTQ3N4sJAeD1KR2Dg4OQyWR485vfDIZhEAwG4XA4cP78echkMtHuV7PUgWEY+Hw+cByHa6+9FjzPw+l04uLFi4hEIjCZTLDZbCv6QfMlmW3/8Ic/zGp0ebFgMmQlUv4jy7KYnZ3F2NgYWltbMTIygsbGxpSBx2QJAXpGoMPhgMPhEF1Wm822Qhho+rmysrLgqmteEPDp/ziDP0+EoJDLYNGr8Mt721FpfN0lp6l0p9OZMZVO4xT5uvX9/f34yEc+gmPHjmHnzp2pHiaNwy0uSW1bEIS0BxbPzMyAZVlxt9Ha2ip2B1C7pjPYbDZbyh7SaDQq2n00GhWLygspcE3G2NgYFhcX0dzcvGL3wfM8PB4PnE4nPB4P9Hq92MpXDE+SEIIHHngAFy5cwCOPPJIuQ1tU285L1Agh6O/vh9/vR0tLCxQKBS5cuICqqqqkZRDx++10q0EkEhFvdCwWE2+0XC5Hb28vmpqaMp5ClQlCiHh6elnVFoRjAurMGijSXBfHcXC5XHA6nfD5fDAajWLFt8/nw8WLF9HW1pZXS8ng4CDuvfde/Od//if27t2b7qGSqBWXvERtdnYW4+PjMBqN2L17t/gcKmi0VGLbtm1ijWMmqH05HA74/f6k2fpcodN6o9Eo9uzZk9ELi3cwnE4nAIi/v3xGiBNC8OCDD+LMmTP49a9/nWkM/vqLGrC0YlkslmXFhNSbEZ9cwIQNjuPgdDoxOzsLj8cDm82GmpoamEymvN1kQRDQ19cHvV6PrVu35j0llKbSFxYWEI1GsWXLFlRVVeUcBL548SI+8IEP4OGHH0Zra2umh0uiVlxSLtgsyyZ9AsuyOHPmjDgDjdo3tW232y2eyZFvnyPN1tvtdrjdbuh0upzjcIQQDAwMQCaTZTU0Ihksy8LpdMLpdCIYDMJkMondO9kI5E9+8hO88MILOH78eDbXvTFELRaLLdsnX7p0CQaDARUVFUtPLEDQKLRkY9++faIX5/F4UFJSIt7obIsOOY5DT08PrFYr6uvrc76WRGjsbdeuXWJbC8uyYtFvporv8fFxvOc978G///u/Y//+/dm8pSRqxSUnUaNTnSsqKhCLxcRjIqltz8zMYHZ2Fi0tLUVrAk/0nmhdWLo4XDEW7mSv6fV6xd+fVqsVt6mJn5UQgp///Oc4deoUHn300WzDMRtT1MbGxqBWq1FdXZ00fpYrtGSjpaVlRXWy3+8XbzQN6NtstpRfIE2519bWoqqqKudrScRut2N8fBzt7e3Lro3neXEb4fP5xIrvxArrqakp3HPPPfi3f/s3XHXVVdm+rSRqxSVrUaMJgZaWFnHmWVNTkxhquXTpEsLhMPbu3buqh/UmxuEsFgvKy8vFOBzP82JpSTEW7mTQg7fp748OmbBarTAYDHj44YfR1dWFkydP5rJz2ZiiNjk5CYZhxDqZfAWNECLOGNu7d29GV5cG9B0OB3ieFwWOxgGS9XEWwsLCAiYnJ9HW1pY2TkC3qU6nEy6XS5zEMTU1hS984Qt48MEHcfjw4VzeWhK14pJR1AghGB8fh9PpFGPHtAbM4XDA6/WC53mUlpauuqAlkhiHMxgM8Pv9WbUlFhPaveNwOPD3f//3GBsbw2OPPYb29vZcXmZjiBrHccvqdWhWiH6h+cS9eJ4XXedM9W7JiMViosCFw2EYDAZ4PB7s2bMHZrM55+tJZG5uDjMzM2hra8u5146K60c+8hEoFAq89tprudakSaJWXFLadjQaFWedMQyTNCFA76fZbIYgCGKNYXl5eV69mIUQiUTw2muvQafTIRwO5xWHK5Suri789Kc/xRe/+EXs3r0bDQ0NuTx944kaIQQLCwuYmJjAli1bsgomJkK3iFVVVUVZabxer9g4HAqFYDQaCyqsnZ2dxdzcnJjCzxWHw4E777wT3/zmN3HttdfmU/ohiVpxSWnbfr8f58+fR3l5Oerr61ckBGjv5K5du8QtKA2L2O1LzewqlQoVFRWrLizhcBjd3d3YsWMHzGZzXnG4Qjl58iQefPBBPPHEE/m2gG0sUYtvdKVZG4/HA4PBIK5amUSETsYoRskGADEL1dLSAp1Ot2xGlcvlgkajEW90NgY3PT0t1iTlI4gulwt33XUXvvrVrxYyAFISteKScvv55z//Gdu2bRPPEIgPpTgcDoyMjIi2FQ8vEAwtBMByAqr0DEKLS2VAtNC2vLy8qOPaA4EAent7sWfPnpQHLGeKwxXKk08+iW9/+9s4depUIbuhjSFq9KDVZBlOGk+iq5ZOpxNHcid6OV6vV5zoQadsFEJ8k3uqLFR8wS8A0eCSrWRTU1NiTCUfQfN6vbjzzjvxP/7H/0BnZ2fOz49DErXiktK2w+EwGIZZ1iFAJ3BQW0iMp8Z4AV99Ygg90z7IZAxK1Ar8f3fsRk2ZVszc2+12Me5bXl5e0ESWxAOFsoGOLrLb7UWph3v22Wfxta99DadOnSo0Xr0xRC0cDovbz3QJATpYzm63w+l0LnPLvV4vxsbG0NLSUpRG39nZWczOzqK1tTVTsZ9IusruyclJeDwetLS05BUj9Pl8uOuuu/DZz34WR48ezfn5CUiiVlxSFt8Gg0HRpul5nIODgyCEYPfu3Ult4Q/9C/j+H8dg0imXpnKEY9hXbcQ3j+xe9jga97Xb7YhEIuIAxlzm/tGdSDYteakotB7uT3/6E7785S/j1KlTYhlXAWwMUfvYxz6G4eFh3H777ejs7ERlZWVWNyUYDMJut2N6ehocx6GxsRFVVVVQq9UIRDn842ND+MuoG3qVHH9/4zbctDe7L2xiYgJutztvjwpYnlGiGcudO3fCkuHU+GQEAgEcPXoUn/jEJ/De9743r+tJQBK14pLUthcWFnDDDTegvb0dnZ2daG1txcTEBKqqqtDQ0JDSxv/95UkcOzsLs35JEKKcAL1Kjoc+lDoLSPsw7XY7AoEATCYTysvLxROfkkHHfOfbwZKMXONwL730Ej7/+c/j8ccfL9ak5o0hatQdP3HiBB599FEQQnDrrbfiyJEj4gEOqZ43PDyMWCyGrVu3ijeVEIJfDBG8OhuBUi4DLyzFMX703ha01KTelhKyNF2D1gkVoyl3dHQUfr8ftbW1cDqdcLvd0Ov14kqWyQsMhUK455578KEPfQgf+tCHCr6ey0iiVlzShlZeeukl/PznP8fJkydx00034bbbbsMNN9yQMtj+lxE3vvGHYRg1S9OaveEYrttuwT+8Y3tWFyMIAjweDxYWFrC4uAij0Yjy8vJlW8O5uTlMT09nLCcqlHRxuNOnT+Ozn/0sHnvssZyP8kvDxhC1ZQ8iBHNzczhx4gROnDiBcDiMW2+9FZ2dnWhsbBQFLl3JBsuyuOH7ryDK8ZAxDGSMDDEB+NRbG/Ghq5N/ecVoB0l8vdHRUYTD4WX9cnQLTVcyOtolWeA3HA7jPe95D+655x589KMfLeh6EpBErbhktO2pqSksLCyA4zgcP34czzzzDJqamnDHHXfgHe94x7JYFiEEv3hlGsfPzQJYGhr6jzfvhEGTe6acJrbit4ZyuRzhcBjt7e1rWg9HC8rtdjseeOABvPjii/jhD39YaHw4kY0naonY7XY8+uijOHHiBDweD26++WYcPnwYTqcTHR0dKUs2On98GnZ/FErZUnV0jCd49w4FOlsrV8Qdit0OQhuAWZbFnj170r5efOM9x3HiSqZQKPCBD3wAt9xyCz75yU8We3yxJGrFJWfbFgQB586dw/Hjx/GHP/wB9fX16OzsxE033SRmH0MsD5YXUKpRFK1FaWhoCG63GwqFQjxdqry8fM1q0ACgu7sb9913H77whS+gqakJhw4dKubLb3xRi8flcuGhhx7CP//zP6OlpQVvetObcOTIkaQB17+OefC5E/3gBQIZw6DBrMVP39eMwKJHjDvQuecTExOw2WxFaQehgyc5jsPu3btzMkZaUT0wMICPfvSjaGpqwqlTp1Zj2KMkasWlINsmhKCvrw/Hjh3DqVOnUF5ejs7OTtxyyy1FKfSm7zE8PAyO48SFNhQKwW63w+FwiLGv8vLyVZuoC7w+Guu3v/0tdu3atRpvcWWJGrC0tRwdHUVlZSUee+wxdHV1YWJiAjfccAPuuOOOZdnFMWcIZye90KsVuH6nBWrF6662IAhYWFjA8PDwshtayOQOQgiGhoYAIO8tbCwWw4c//GF0dHTg0KFDuOGGG1bjkAlJ1IpL0Y43ouOsjh8/jscffxylpaW4/fbbcdttt4ljtHOFnnmgVCqxffv2lDPZqMBxHCdmUvMZFZQKOhrrV7/6Ffbt21eU10zClSdqyfD7/XjiiSfQ1dWF4eFhXH/99Thy5Eja80Fpa8q2bdtgNpvh9XqXFftWVFTkVHNDY3IKhSKl4WSC4zh87GMfw969e/HlL395NU/MkUStuKyKbdPE1fHjx3Hy5EloNBrcdttt6OzsREVFRVb2IQiC2A2zZcuWrJ5DD/q22+0Ih8NFKbLNcTRWIWwOUYsnFArhySefxIkTJ9DT04PrrrsOR44cwVVXXSUKVKpzCoHlgVWXywW9Xp+y2Df+ORcuXIBKpUJTU1NeN57neXzyk59EQ0MDvva1r632EWCSqBWXVbdtQggmJibQ1dWF3/3udwCA2267DUeOHEFNTU1Se6EjsujU3HyID+77/X5xZHcuO5o8RmMVwuYTtXgikQieeeYZHD9+HGfPnsWb3/xm8UzPd73rXRmH76Uq9o0/A0EQBFy4cAFarTbvJIMgCPjMZz4Ds9mMb33rW0UpJcmAJGrFZU1tmxCC2dlZdHV14dFHH0U0GhUrBKg3FovFcP78+aKNyAJeLxWx2+3wer1ZtS/mORqrEDa3qMXDsiweeughfPnLX0ZDQwNaWlrQ2dmJa6+9NuvMDy32dTgcYimGy+VCaWkptm7dmtd1CYKA+++/HyqVCt/97nfXQtAASdSKzbrZNiFErBDo6urC4uIi3va2t4FlWXzmM58pyoisVO8b376o1WrFHQ1d8GdnZ3H33Xfj+9//Pt7ylresynUk4Y0jasCSO+7xeGAymfBf//VfOHbsGP785z+jo6MDnZ2duP7667OurA6FQuju7gbP81Cr1eJp17lkjgRBwJe+9CWwLIv/83/+z1oJGiCJWrFZd9umLCws4PDhw2hoaIDX68W73vUusUJgtUIatIuA7mgUCgVOnz6NX/ziF/jud7+L66+/flXeNwVvLFFLBq34Pn78OP74xz9i7969OHLkSNqKb0EQ0NPTA7PZjPr6erFq2m63g+O4rJqMBUHAP/3TP8HtduOnP/3pWgoaIIlasdlQth0Oh6HVauH1enHy5El0dXVhcnISN954I+644w40Nzevqr1NT0/j3nvvRTAYxIMPPohrr7121d4rCZKoxSMIAk6fPo1jx47h2WefTVrxzfO8eD5BsuBrNk3GhBD88z//MyYnJ/HQQw+taVX3ZSRRKy4b3rZ9Pp9YIXDx4kW8/e1vx5EjR9DR0VFUgaOjsf7n//yfuOWWW4r2ujkgiVoqaMX3sWPH8NRTT6G+vh7vfOc7sbi4iKNHj6Kmpibja9Cm9oWFBQSDQVgsFgDA7373OwwNDeHhhx9e06mmcUiiVlyuKNsOhUI4deoUurq60NfXh7e+9a3o7OxcViGQD3Q01he+8AUcOXKkeBecG5KoZQOt9bnrrrvElPbtt9+OW2+9NeuKbzqP/h/+4R/wzDPP4Dvf+Q7uvffe1b3w1EiiVlyuWNuORCJ4+umncfz4cbz22ms4fPgwjhw5gje96U05Lbg+nw9Hjx7Fpz/9adx9992reMUZkUQtF6amplBbW5tXxXf8+YW//vWvEYlEitYCkweSqBWXK962gaUKgeeeew5dXV145ZVXcNVVV+HIkSO49tpr007yCAQCuPvuu3Hffffhfe973xpecVIkUSuExIpvtVotzoSLr/im5xfSouBijmHOE0nUisums+1YLCZWCLz44ovo6OjAkSNH8La3vW1ZhQAdjfXBD35wPXce8UiiVizSVXw///zz+ZxfuJpIolZcNrVt8zyPF198UawQaG5uxpEjR3D48GH8t//233D06FF87GMfW+/LpEiithrEV3z/x3/8B5xOJ3p6egqaI19kJFErLm8Y2xYEAa+88gqOHz+ORx55BB//+MfxT//0T6vd1pcLkqitNoQQxGKxNZ1XlQUbxgI3CW9I22ZZFkqlciMJGiCJ2huWDWWFmwDJtjcORbXtNS2Jl5CQkFhtJFGTkJDYVEiiJiEhsamQRE1CQmJTkamnQgpOS2xWJNvepEiemoSExKZCEjUJCYlNhSRqEhISmwpJ1CQkJDYVkqhJSEhsKiRRk5CQ2FRIoiYhIbGpkERNQkJiUyGJmoSExKZCEjUJCYlNhSRqEhISmwpJ1CQkJDYV/z/uemWnc2Su6gAAAABJRU5ErkJggg==",
      "image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<!-- Created with matplotlib (https://matplotlib.org/) -->\n<svg height=\"252.766926pt\" version=\"1.1\" viewBox=\"0 0 309.080685 252.766926\" width=\"309.080685pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n <metadata>\n  <rdf:RDF xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n   <cc:Work>\n    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n    <dc:date>2021-08-21T09:59:53.266330</dc:date>\n    <dc:format>image/svg+xml</dc:format>\n    <dc:creator>\n     <cc:Agent>\n      <dc:title>Matplotlib v3.3.4, https://matplotlib.org/</dc:title>\n     </cc:Agent>\n    </dc:creator>\n   </cc:Work>\n  </rdf:RDF>\n </metadata>\n <defs>\n  <style type=\"text/css\">*{stroke-linecap:butt;stroke-linejoin:round;}</style>\n </defs>\n <g id=\"figure_1\">\n  <g id=\"patch_1\">\n   <path d=\"M 0 252.766926 \nL 309.080685 252.766926 \nL 309.080685 0 \nL 0 0 \nz\n\" style=\"fill:none;\"/>\n  </g>\n  <g id=\"patch_2\">\n   <path d=\"M 7.2 120.942614 \nL 106.036364 120.942614 \nL 106.036364 22.10625 \nL 7.2 22.10625 \nz\n\" style=\"fill:#ffffff;\"/>\n  </g>\n  <g id=\"pane3d_1\">\n   <g id=\"patch_3\">\n    <path d=\"M 14.662827 96.572734 \nL 47.301822 69.214081 \nL 46.848109 29.758013 \nL 12.64717 54.7163 \n\" style=\"fill:#f2f2f2;opacity:0.5;stroke:#f2f2f2;stroke-linejoin:miter;\"/>\n   </g>\n  </g>\n  <g id=\"pane3d_2\">\n   <g id=\"patch_4\">\n    <path d=\"M 47.301822 69.214081 \nL 99.675764 84.437118 \nL 101.544805 43.622031 \nL 46.848109 29.758013 \n\" style=\"fill:#e6e6e6;opacity:0.5;stroke:#e6e6e6;stroke-linejoin:miter;\"/>\n   </g>\n  </g>\n  <g id=\"pane3d_3\">\n   <g id=\"patch_5\">\n    <path d=\"M 14.662827 96.572734 \nL 70.181763 114.705218 \nL 99.675764 84.437118 \nL 47.301822 69.214081 \n\" style=\"fill:#ececec;opacity:0.5;stroke:#ececec;stroke-linejoin:miter;\"/>\n   </g>\n  </g>\n  <g id=\"axis3d_1\">\n   <g id=\"line2d_1\">\n    <path d=\"M 14.662827 96.572734 \nL 70.181763 114.705218 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"Line3DCollection_1\">\n    <path d=\"M 26.773964 100.528231 \nL 58.766384 72.546376 \nL 58.801386 32.78782 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 41.690472 105.399962 \nL 72.856197 76.641729 \nL 73.506956 36.515253 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 57.042232 110.413847 \nL 87.322276 80.846447 \nL 88.622624 40.346634 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"xtick_1\">\n    <g id=\"line2d_2\">\n     <path d=\"M 27.052745 100.284398 \nL 26.215196 101.016952 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_2\">\n    <g id=\"line2d_3\">\n     <path d=\"M 41.962368 105.149069 \nL 41.145485 105.90285 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_3\">\n    <g id=\"line2d_4\">\n     <path d=\"M 57.306721 110.155583 \nL 56.512075 110.931527 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n  </g>\n  <g id=\"axis3d_2\">\n   <g id=\"line2d_5\">\n    <path d=\"M 99.675764 84.437118 \nL 70.181763 114.705218 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"Line3DCollection_2\">\n    <path d=\"M 15.012249 52.990373 \nL 16.91193 94.687491 \nL 72.22252 112.6109 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 34.196256 38.990759 \nL 35.198873 79.359013 \nL 88.769382 95.629749 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"xtick_4\">\n    <g id=\"line2d_6\">\n     <path d=\"M 71.756416 112.459859 \nL 73.155933 112.913373 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_5\">\n    <g id=\"line2d_7\">\n     <path d=\"M 88.319048 95.492971 \nL 89.671157 95.903641 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n  </g>\n  <g id=\"axis3d_3\">\n   <g id=\"line2d_8\">\n    <path d=\"M 99.675764 84.437118 \nL 101.544805 43.622031 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"Line3DCollection_3\">\n    <path d=\"M 100.086333 75.471324 \nL 47.201991 60.532535 \nL 14.220633 87.390281 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 100.589811 64.476664 \nL 47.079696 49.897436 \nL 13.677927 76.120649 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 101.105467 53.216057 \nL 46.954587 39.017618 \nL 13.121583 64.567801 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"xtick_6\">\n    <g id=\"line2d_9\">\n     <path d=\"M 99.642284 75.345889 \nL 100.975499 75.722496 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_7\">\n    <g id=\"line2d_10\">\n     <path d=\"M 100.140253 64.354179 \nL 101.490018 64.721932 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_8\">\n    <g id=\"line2d_11\">\n     <path d=\"M 100.650263 53.096702 \nL 102.016994 53.45506 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n  </g>\n  <g id=\"axes_1\">\n   <g id=\"Path3DCollection_1\">\n    <defs>\n     <path d=\"M 0 2.236068 \nC 0.593012 2.236068 1.161816 2.000462 1.581139 1.581139 \nC 2.000462 1.161816 2.236068 0.593012 2.236068 -0 \nC 2.236068 -0.593012 2.000462 -1.161816 1.581139 -1.581139 \nC 1.161816 -2.000462 0.593012 -2.236068 0 -2.236068 \nC -0.593012 -2.236068 -1.161816 -2.000462 -1.581139 -1.581139 \nC -2.000462 -1.161816 -2.236068 -0.593012 -2.236068 0 \nC -2.236068 0.593012 -2.000462 1.161816 -1.581139 1.581139 \nC -1.161816 2.000462 -0.593012 2.236068 0 2.236068 \nz\n\" id=\"C0_0_843c080b3b\"/>\n    </defs>\n    <g clip-path=\"url(#pa8abc3a947)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.3;stroke:#1f77b4;stroke-opacity:0.3;\" x=\"44.875213\" xlink:href=\"#C0_0_843c080b3b\" y=\"46.229871\"/>\n    </g>\n    <g clip-path=\"url(#pa8abc3a947)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.338572;stroke:#1f77b4;stroke-opacity:0.338572;\" x=\"88.727318\" xlink:href=\"#C0_0_843c080b3b\" y=\"76.657335\"/>\n    </g>\n    <g clip-path=\"url(#pa8abc3a947)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.536217;stroke:#1f77b4;stroke-opacity:0.536217;\" x=\"55.44058\" xlink:href=\"#C0_0_843c080b3b\" y=\"61.039286\"/>\n    </g>\n    <g clip-path=\"url(#pa8abc3a947)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.553744;stroke:#1f77b4;stroke-opacity:0.553744;\" x=\"73.314023\" xlink:href=\"#C0_0_843c080b3b\" y=\"79.589328\"/>\n    </g>\n    <g clip-path=\"url(#pa8abc3a947)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.597695;stroke:#1f77b4;stroke-opacity:0.597695;\" x=\"82.853426\" xlink:href=\"#C0_0_843c080b3b\" y=\"69.385879\"/>\n    </g>\n    <g clip-path=\"url(#pa8abc3a947)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.618261;stroke:#1f77b4;stroke-opacity:0.618261;\" x=\"51.015489\" xlink:href=\"#C0_0_843c080b3b\" y=\"94.22385\"/>\n    </g>\n    <g clip-path=\"url(#pa8abc3a947)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.800065;stroke:#1f77b4;stroke-opacity:0.800065;\" x=\"89.639925\" xlink:href=\"#C0_0_843c080b3b\" y=\"52.698654\"/>\n    </g>\n    <g clip-path=\"url(#pa8abc3a947)\">\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"62.303957\" xlink:href=\"#C0_0_843c080b3b\" y=\"97.690815\"/>\n    </g>\n   </g>\n   <g id=\"text_1\">\n    <!-- c_0 -->\n    <g transform=\"translate(46.501619 16.10625)scale(0.12 -0.12)\">\n     <defs>\n      <path d=\"M 48.78125 52.59375 \nL 48.78125 44.1875 \nQ 44.96875 46.296875 41.140625 47.34375 \nQ 37.3125 48.390625 33.40625 48.390625 \nQ 24.65625 48.390625 19.8125 42.84375 \nQ 14.984375 37.3125 14.984375 27.296875 \nQ 14.984375 17.28125 19.8125 11.734375 \nQ 24.65625 6.203125 33.40625 6.203125 \nQ 37.3125 6.203125 41.140625 7.25 \nQ 44.96875 8.296875 48.78125 10.40625 \nL 48.78125 2.09375 \nQ 45.015625 0.34375 40.984375 -0.53125 \nQ 36.96875 -1.421875 32.421875 -1.421875 \nQ 20.0625 -1.421875 12.78125 6.34375 \nQ 5.515625 14.109375 5.515625 27.296875 \nQ 5.515625 40.671875 12.859375 48.328125 \nQ 20.21875 56 33.015625 56 \nQ 37.15625 56 41.109375 55.140625 \nQ 45.0625 54.296875 48.78125 52.59375 \nz\n\" id=\"DejaVuSans-99\"/>\n      <path d=\"M 50.984375 -16.609375 \nL 50.984375 -23.578125 \nL -0.984375 -23.578125 \nL -0.984375 -16.609375 \nz\n\" id=\"DejaVuSans-95\"/>\n      <path d=\"M 31.78125 66.40625 \nQ 24.171875 66.40625 20.328125 58.90625 \nQ 16.5 51.421875 16.5 36.375 \nQ 16.5 21.390625 20.328125 13.890625 \nQ 24.171875 6.390625 31.78125 6.390625 \nQ 39.453125 6.390625 43.28125 13.890625 \nQ 47.125 21.390625 47.125 36.375 \nQ 47.125 51.421875 43.28125 58.90625 \nQ 39.453125 66.40625 31.78125 66.40625 \nz\nM 31.78125 74.21875 \nQ 44.046875 74.21875 50.515625 64.515625 \nQ 56.984375 54.828125 56.984375 36.375 \nQ 56.984375 17.96875 50.515625 8.265625 \nQ 44.046875 -1.421875 31.78125 -1.421875 \nQ 19.53125 -1.421875 13.0625 8.265625 \nQ 6.59375 17.96875 6.59375 36.375 \nQ 6.59375 54.828125 13.0625 64.515625 \nQ 19.53125 74.21875 31.78125 74.21875 \nz\n\" id=\"DejaVuSans-48\"/>\n     </defs>\n     <use xlink:href=\"#DejaVuSans-99\"/>\n     <use x=\"54.980469\" xlink:href=\"#DejaVuSans-95\"/>\n     <use x=\"104.980469\" xlink:href=\"#DejaVuSans-48\"/>\n    </g>\n   </g>\n  </g>\n  <g id=\"patch_6\">\n   <path d=\"M 189.818182 120.942614 \nL 288.654545 120.942614 \nL 288.654545 22.10625 \nL 189.818182 22.10625 \nz\n\" style=\"fill:#ffffff;\"/>\n  </g>\n  <g id=\"pane3d_4\">\n   <g id=\"patch_7\">\n    <path d=\"M 197.281009 96.572734 \nL 229.920004 69.214081 \nL 229.466291 29.758013 \nL 195.265351 54.7163 \n\" style=\"fill:#f2f2f2;opacity:0.5;stroke:#f2f2f2;stroke-linejoin:miter;\"/>\n   </g>\n  </g>\n  <g id=\"pane3d_5\">\n   <g id=\"patch_8\">\n    <path d=\"M 229.920004 69.214081 \nL 282.293946 84.437118 \nL 284.162987 43.622031 \nL 229.466291 29.758013 \n\" style=\"fill:#e6e6e6;opacity:0.5;stroke:#e6e6e6;stroke-linejoin:miter;\"/>\n   </g>\n  </g>\n  <g id=\"pane3d_6\">\n   <g id=\"patch_9\">\n    <path d=\"M 197.281009 96.572734 \nL 252.799945 114.705218 \nL 282.293946 84.437118 \nL 229.920004 69.214081 \n\" style=\"fill:#ececec;opacity:0.5;stroke:#ececec;stroke-linejoin:miter;\"/>\n   </g>\n  </g>\n  <g id=\"axis3d_4\">\n   <g id=\"line2d_12\">\n    <path d=\"M 197.281009 96.572734 \nL 252.799945 114.705218 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"Line3DCollection_4\">\n    <path d=\"M 200.643527 97.670932 \nL 233.105227 70.139899 \nL 232.786187 30.59951 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 227.005504 106.280754 \nL 258.018198 77.381117 \nL 258.781908 37.188667 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"xtick_9\">\n    <g id=\"line2d_13\">\n     <path d=\"M 200.926202 97.431194 \nL 200.076965 98.151438 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_10\">\n    <g id=\"line2d_14\">\n     <path d=\"M 227.276123 106.028573 \nL 226.463075 106.786225 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n  </g>\n  <g id=\"axis3d_5\">\n   <g id=\"line2d_15\">\n    <path d=\"M 282.293946 84.437118 \nL 252.799945 114.705218 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"Line3DCollection_5\">\n    <path d=\"M 208.096673 45.352586 \nL 209.49729 86.332805 \nL 263.869615 103.345013 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 220.432038 36.350799 \nL 221.27423 76.46114 \nL 274.506582 92.428868 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"xtick_11\">\n    <g id=\"line2d_16\">\n     <path d=\"M 263.412031 103.201843 \nL 264.785936 103.631715 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_12\">\n    <g id=\"line2d_17\">\n     <path d=\"M 274.059299 92.2947 \nL 275.40224 92.697532 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n  </g>\n  <g id=\"axis3d_6\">\n   <g id=\"line2d_18\">\n    <path d=\"M 282.293946 84.437118 \nL 284.162987 43.622031 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"Line3DCollection_6\">\n    <path d=\"M 282.408096 81.944358 \nL 229.892239 66.799541 \nL 197.158098 94.020418 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 283.298762 62.49449 \nL 229.675845 47.981372 \nL 196.198215 74.087811 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"xtick_13\">\n    <g id=\"line2d_19\">\n     <path d=\"M 281.967287 81.817235 \nL 283.290766 82.198907 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_14\">\n    <g id=\"line2d_20\">\n     <path d=\"M 282.848211 62.372547 \nL 284.200962 62.738671 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n  </g>\n  <g id=\"axes_2\">\n   <g id=\"Path3DCollection_2\">\n    <defs>\n     <path d=\"M 0 2.236068 \nC 0.593012 2.236068 1.161816 2.000462 1.581139 1.581139 \nC 2.000462 1.161816 2.236068 0.593012 2.236068 -0 \nC 2.236068 -0.593012 2.000462 -1.161816 1.581139 -1.581139 \nC 1.161816 -2.000462 0.593012 -2.236068 0 -2.236068 \nC -0.593012 -2.236068 -1.161816 -2.000462 -1.581139 -1.581139 \nC -2.000462 -1.161816 -2.236068 -0.593012 -2.236068 0 \nC -2.236068 0.593012 -2.000462 1.161816 -1.581139 1.581139 \nC -1.161816 2.000462 -0.593012 2.236068 0 2.236068 \nz\n\" id=\"C1_0_890bb03686\"/>\n    </defs>\n    <g clip-path=\"url(#p499016b9a9)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.3;stroke:#1f77b4;stroke-opacity:0.3;\" x=\"230.822804\" xlink:href=\"#C1_0_890bb03686\" y=\"34.69646\"/>\n    </g>\n    <g clip-path=\"url(#p499016b9a9)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.335815;stroke:#1f77b4;stroke-opacity:0.335815;\" x=\"271.992042\" xlink:href=\"#C1_0_890bb03686\" y=\"77.326823\"/>\n    </g>\n    <g clip-path=\"url(#p499016b9a9)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.337905;stroke:#1f77b4;stroke-opacity:0.337905;\" x=\"236.613981\" xlink:href=\"#C1_0_890bb03686\" y=\"56.80548\"/>\n    </g>\n    <g clip-path=\"url(#p499016b9a9)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.428807;stroke:#1f77b4;stroke-opacity:0.428807;\" x=\"273.111019\" xlink:href=\"#C1_0_890bb03686\" y=\"86.761218\"/>\n    </g>\n    <g clip-path=\"url(#p499016b9a9)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.505378;stroke:#1f77b4;stroke-opacity:0.505378;\" x=\"253.059586\" xlink:href=\"#C1_0_890bb03686\" y=\"74.453399\"/>\n    </g>\n    <g clip-path=\"url(#p499016b9a9)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.583628;stroke:#1f77b4;stroke-opacity:0.583628;\" x=\"237.189253\" xlink:href=\"#C1_0_890bb03686\" y=\"45.868339\"/>\n    </g>\n    <g clip-path=\"url(#p499016b9a9)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.994605;stroke:#1f77b4;stroke-opacity:0.994605;\" x=\"234.379103\" xlink:href=\"#C1_0_890bb03686\" y=\"78.975625\"/>\n    </g>\n    <g clip-path=\"url(#p499016b9a9)\">\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"244.906105\" xlink:href=\"#C1_0_890bb03686\" y=\"67.912611\"/>\n    </g>\n   </g>\n   <g id=\"text_2\">\n    <!-- c_1 -->\n    <g transform=\"translate(229.119801 16.10625)scale(0.12 -0.12)\">\n     <defs>\n      <path d=\"M 12.40625 8.296875 \nL 28.515625 8.296875 \nL 28.515625 63.921875 \nL 10.984375 60.40625 \nL 10.984375 69.390625 \nL 28.421875 72.90625 \nL 38.28125 72.90625 \nL 38.28125 8.296875 \nL 54.390625 8.296875 \nL 54.390625 0 \nL 12.40625 0 \nz\n\" id=\"DejaVuSans-49\"/>\n     </defs>\n     <use xlink:href=\"#DejaVuSans-99\"/>\n     <use x=\"54.980469\" xlink:href=\"#DejaVuSans-95\"/>\n     <use x=\"104.980469\" xlink:href=\"#DejaVuSans-49\"/>\n    </g>\n   </g>\n  </g>\n  <g id=\"patch_10\">\n   <path d=\"M 7.2 239.54625 \nL 106.036364 239.54625 \nL 106.036364 140.709886 \nL 7.2 140.709886 \nz\n\" style=\"fill:#ffffff;\"/>\n  </g>\n  <g id=\"pane3d_7\">\n   <g id=\"patch_11\">\n    <path d=\"M 14.662827 215.17637 \nL 47.301822 187.817717 \nL 46.848109 148.361649 \nL 12.64717 173.319937 \n\" style=\"fill:#f2f2f2;opacity:0.5;stroke:#f2f2f2;stroke-linejoin:miter;\"/>\n   </g>\n  </g>\n  <g id=\"pane3d_8\">\n   <g id=\"patch_12\">\n    <path d=\"M 47.301822 187.817717 \nL 99.675764 203.040754 \nL 101.544805 162.225667 \nL 46.848109 148.361649 \n\" style=\"fill:#e6e6e6;opacity:0.5;stroke:#e6e6e6;stroke-linejoin:miter;\"/>\n   </g>\n  </g>\n  <g id=\"pane3d_9\">\n   <g id=\"patch_13\">\n    <path d=\"M 14.662827 215.17637 \nL 70.181763 233.308855 \nL 99.675764 203.040754 \nL 47.301822 187.817717 \n\" style=\"fill:#ececec;opacity:0.5;stroke:#ececec;stroke-linejoin:miter;\"/>\n   </g>\n  </g>\n  <g id=\"axis3d_7\">\n   <g id=\"line2d_21\">\n    <path d=\"M 14.662827 215.17637 \nL 70.181763 233.308855 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"Line3DCollection_7\">\n    <path d=\"M 26.773964 219.131867 \nL 58.766384 191.150013 \nL 58.801386 151.391456 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 41.690472 224.003599 \nL 72.856197 195.245365 \nL 73.506956 155.11889 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 57.042232 229.017484 \nL 87.322276 199.450083 \nL 88.622624 158.950271 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"xtick_15\">\n    <g id=\"line2d_22\">\n     <path d=\"M 27.052745 218.888035 \nL 26.215196 219.620588 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_16\">\n    <g id=\"line2d_23\">\n     <path d=\"M 41.962368 223.752706 \nL 41.145485 224.506487 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_17\">\n    <g id=\"line2d_24\">\n     <path d=\"M 57.306721 228.759219 \nL 56.512075 229.535163 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n  </g>\n  <g id=\"axis3d_8\">\n   <g id=\"line2d_25\">\n    <path d=\"M 99.675764 203.040754 \nL 70.181763 233.308855 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"Line3DCollection_8\">\n    <path d=\"M 17.302199 169.922908 \nL 19.09071 211.464831 \nL 74.198283 229.186918 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 28.368771 161.847028 \nL 29.635651 202.625852 \nL 83.74419 219.390468 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 38.834836 154.20937 \nL 39.632256 194.2465 \nL 92.768595 210.129209 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"xtick_18\">\n    <g id=\"line2d_26\">\n     <path d=\"M 73.734025 229.037617 \nL 75.127991 229.485903 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_19\">\n    <g id=\"line2d_27\">\n     <path d=\"M 83.288994 219.249434 \nL 84.655719 219.67289 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_20\">\n    <g id=\"line2d_28\">\n     <path d=\"M 92.322176 209.995772 \nL 93.662517 210.396407 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n  </g>\n  <g id=\"axis3d_9\">\n   <g id=\"line2d_29\">\n    <path d=\"M 99.675764 203.040754 \nL 101.544805 162.225667 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"Line3DCollection_9\">\n    <path d=\"M 100.086333 194.074961 \nL 47.201991 179.136171 \nL 14.220633 205.993917 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 100.589811 183.0803 \nL 47.079696 168.501072 \nL 13.677927 194.724285 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 101.105467 171.819693 \nL 46.954587 157.621254 \nL 13.121583 183.171438 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"xtick_21\">\n    <g id=\"line2d_30\">\n     <path d=\"M 99.642284 193.949525 \nL 100.975499 194.326133 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_22\">\n    <g id=\"line2d_31\">\n     <path d=\"M 100.140253 182.957815 \nL 101.490018 183.325569 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_23\">\n    <g id=\"line2d_32\">\n     <path d=\"M 100.650263 171.700338 \nL 102.016994 172.058697 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n  </g>\n  <g id=\"axes_3\">\n   <g id=\"Path3DCollection_3\">\n    <defs>\n     <path d=\"M 0 2.236068 \nC 0.593012 2.236068 1.161816 2.000462 1.581139 1.581139 \nC 2.000462 1.161816 2.236068 0.593012 2.236068 -0 \nC 2.236068 -0.593012 2.000462 -1.161816 1.581139 -1.581139 \nC 1.161816 -2.000462 0.593012 -2.236068 0 -2.236068 \nC -0.593012 -2.236068 -1.161816 -2.000462 -1.581139 -1.581139 \nC -2.000462 -1.161816 -2.236068 -0.593012 -2.236068 0 \nC -2.236068 0.593012 -2.000462 1.161816 -1.581139 1.581139 \nC -1.161816 2.000462 -0.593012 2.236068 0 2.236068 \nz\n\" id=\"C2_0_f6092997d1\"/>\n    </defs>\n    <g clip-path=\"url(#p85e40e64a7)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.3;stroke:#1f77b4;stroke-opacity:0.3;\" x=\"65.450919\" xlink:href=\"#C2_0_f6092997d1\" y=\"179.982599\"/>\n    </g>\n    <g clip-path=\"url(#p85e40e64a7)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.470232;stroke:#1f77b4;stroke-opacity:0.470232;\" x=\"44.253216\" xlink:href=\"#C2_0_f6092997d1\" y=\"156.247671\"/>\n    </g>\n    <g clip-path=\"url(#p85e40e64a7)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.519593;stroke:#1f77b4;stroke-opacity:0.519593;\" x=\"85.195434\" xlink:href=\"#C2_0_f6092997d1\" y=\"198.679155\"/>\n    </g>\n    <g clip-path=\"url(#p85e40e64a7)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.724022;stroke:#1f77b4;stroke-opacity:0.724022;\" x=\"40.06589\" xlink:href=\"#C2_0_f6092997d1\" y=\"199.44111\"/>\n    </g>\n    <g clip-path=\"url(#p85e40e64a7)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.764576;stroke:#1f77b4;stroke-opacity:0.764576;\" x=\"33.984541\" xlink:href=\"#C2_0_f6092997d1\" y=\"188.739078\"/>\n    </g>\n    <g clip-path=\"url(#p85e40e64a7)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.810069;stroke:#1f77b4;stroke-opacity:0.810069;\" x=\"92.05094\" xlink:href=\"#C2_0_f6092997d1\" y=\"178.086514\"/>\n    </g>\n    <g clip-path=\"url(#p85e40e64a7)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.905327;stroke:#1f77b4;stroke-opacity:0.905327;\" x=\"37.788984\" xlink:href=\"#C2_0_f6092997d1\" y=\"217.49946\"/>\n    </g>\n    <g clip-path=\"url(#p85e40e64a7)\">\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"64.167579\" xlink:href=\"#C2_0_f6092997d1\" y=\"202.228572\"/>\n    </g>\n   </g>\n   <g id=\"text_3\">\n    <!-- c_2 -->\n    <g transform=\"translate(46.501619 134.709886)scale(0.12 -0.12)\">\n     <defs>\n      <path d=\"M 19.1875 8.296875 \nL 53.609375 8.296875 \nL 53.609375 0 \nL 7.328125 0 \nL 7.328125 8.296875 \nQ 12.9375 14.109375 22.625 23.890625 \nQ 32.328125 33.6875 34.8125 36.53125 \nQ 39.546875 41.84375 41.421875 45.53125 \nQ 43.3125 49.21875 43.3125 52.78125 \nQ 43.3125 58.59375 39.234375 62.25 \nQ 35.15625 65.921875 28.609375 65.921875 \nQ 23.96875 65.921875 18.8125 64.3125 \nQ 13.671875 62.703125 7.8125 59.421875 \nL 7.8125 69.390625 \nQ 13.765625 71.78125 18.9375 73 \nQ 24.125 74.21875 28.421875 74.21875 \nQ 39.75 74.21875 46.484375 68.546875 \nQ 53.21875 62.890625 53.21875 53.421875 \nQ 53.21875 48.921875 51.53125 44.890625 \nQ 49.859375 40.875 45.40625 35.40625 \nQ 44.1875 33.984375 37.640625 27.21875 \nQ 31.109375 20.453125 19.1875 8.296875 \nz\n\" id=\"DejaVuSans-50\"/>\n     </defs>\n     <use xlink:href=\"#DejaVuSans-99\"/>\n     <use x=\"54.980469\" xlink:href=\"#DejaVuSans-95\"/>\n     <use x=\"104.980469\" xlink:href=\"#DejaVuSans-50\"/>\n    </g>\n   </g>\n  </g>\n  <g id=\"patch_14\">\n   <path d=\"M 189.818182 239.54625 \nL 288.654545 239.54625 \nL 288.654545 140.709886 \nL 189.818182 140.709886 \nz\n\" style=\"fill:#ffffff;\"/>\n  </g>\n  <g id=\"pane3d_10\">\n   <g id=\"patch_15\">\n    <path d=\"M 197.281009 215.17637 \nL 229.920004 187.817717 \nL 229.466291 148.361649 \nL 195.265351 173.319937 \n\" style=\"fill:#f2f2f2;opacity:0.5;stroke:#f2f2f2;stroke-linejoin:miter;\"/>\n   </g>\n  </g>\n  <g id=\"pane3d_11\">\n   <g id=\"patch_16\">\n    <path d=\"M 229.920004 187.817717 \nL 282.293946 203.040754 \nL 284.162987 162.225667 \nL 229.466291 148.361649 \n\" style=\"fill:#e6e6e6;opacity:0.5;stroke:#e6e6e6;stroke-linejoin:miter;\"/>\n   </g>\n  </g>\n  <g id=\"pane3d_12\">\n   <g id=\"patch_17\">\n    <path d=\"M 197.281009 215.17637 \nL 252.799945 233.308855 \nL 282.293946 203.040754 \nL 229.920004 187.817717 \n\" style=\"fill:#ececec;opacity:0.5;stroke:#ececec;stroke-linejoin:miter;\"/>\n   </g>\n  </g>\n  <g id=\"axis3d_10\">\n   <g id=\"line2d_33\">\n    <path d=\"M 197.281009 215.17637 \nL 252.799945 233.308855 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"Line3DCollection_10\">\n    <path d=\"M 200.643527 216.274569 \nL 233.105227 188.743536 \nL 232.786187 149.203146 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 227.005504 224.88439 \nL 258.018198 195.984753 \nL 258.781908 155.792304 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"xtick_24\">\n    <g id=\"line2d_34\">\n     <path d=\"M 200.926202 216.03483 \nL 200.076965 216.755075 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_25\">\n    <g id=\"line2d_35\">\n     <path d=\"M 227.276123 224.63221 \nL 226.463075 225.389861 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n  </g>\n  <g id=\"axis3d_11\">\n   <g id=\"line2d_36\">\n    <path d=\"M 282.293946 203.040754 \nL 252.799945 233.308855 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"Line3DCollection_11\">\n    <path d=\"M 203.586021 167.247891 \nL 205.198882 208.539452 \nL 259.978829 225.941553 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 213.126608 160.285605 \nL 214.295623 200.914383 \nL 268.207575 217.496836 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 222.215125 153.653219 \nL 222.979268 193.63558 \nL 276.043773 209.454969 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"xtick_26\">\n    <g id=\"line2d_37\">\n     <path d=\"M 259.517548 225.795017 \nL 260.902566 226.234999 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_27\">\n    <g id=\"line2d_38\">\n     <path d=\"M 267.754157 217.357372 \nL 269.115537 217.776111 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_28\">\n    <g id=\"line2d_39\">\n     <path d=\"M 275.598001 209.322077 \nL 276.936397 209.721075 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n  </g>\n  <g id=\"axis3d_12\">\n   <g id=\"line2d_40\">\n    <path d=\"M 282.293946 203.040754 \nL 284.162987 162.225667 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"Line3DCollection_12\">\n    <path d=\"M 282.408096 200.547994 \nL 229.892239 185.403177 \nL 197.158098 212.624054 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n    <path d=\"M 283.298762 181.098126 \nL 229.675845 166.585008 \nL 196.198215 192.691448 \n\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\n   </g>\n   <g id=\"xtick_29\">\n    <g id=\"line2d_41\">\n     <path d=\"M 281.967287 200.420871 \nL 283.290766 200.802543 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n   <g id=\"xtick_30\">\n    <g id=\"line2d_42\">\n     <path d=\"M 282.848211 180.976184 \nL 284.200962 181.342307 \n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\n    </g>\n   </g>\n  </g>\n  <g id=\"axes_4\">\n   <g id=\"Path3DCollection_4\">\n    <defs>\n     <path d=\"M 0 2.236068 \nC 0.593012 2.236068 1.161816 2.000462 1.581139 1.581139 \nC 2.000462 1.161816 2.236068 0.593012 2.236068 -0 \nC 2.236068 -0.593012 2.000462 -1.161816 1.581139 -1.581139 \nC 1.161816 -2.000462 0.593012 -2.236068 0 -2.236068 \nC -0.593012 -2.236068 -1.161816 -2.000462 -1.581139 -1.581139 \nC -2.000462 -1.161816 -2.236068 -0.593012 -2.236068 0 \nC -2.236068 0.593012 -2.000462 1.161816 -1.581139 1.581139 \nC -1.161816 2.000462 -0.593012 2.236068 0 2.236068 \nz\n\" id=\"C3_0_a2946a6d36\"/>\n    </defs>\n    <g clip-path=\"url(#pcba1dba992)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.3;stroke:#1f77b4;stroke-opacity:0.3;\" x=\"231.059918\" xlink:href=\"#C3_0_a2946a6d36\" y=\"176.669125\"/>\n    </g>\n    <g clip-path=\"url(#pcba1dba992)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.465018;stroke:#1f77b4;stroke-opacity:0.465018;\" x=\"246.211559\" xlink:href=\"#C3_0_a2946a6d36\" y=\"169.235009\"/>\n    </g>\n    <g clip-path=\"url(#pcba1dba992)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.697033;stroke:#1f77b4;stroke-opacity:0.697033;\" x=\"216.609161\" xlink:href=\"#C3_0_a2946a6d36\" y=\"163.902847\"/>\n    </g>\n    <g clip-path=\"url(#pcba1dba992)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.734145;stroke:#1f77b4;stroke-opacity:0.734145;\" x=\"227.183645\" xlink:href=\"#C3_0_a2946a6d36\" y=\"170.869797\"/>\n    </g>\n    <g clip-path=\"url(#pcba1dba992)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.736509;stroke:#1f77b4;stroke-opacity:0.736509;\" x=\"278.598272\" xlink:href=\"#C3_0_a2946a6d36\" y=\"165.617114\"/>\n    </g>\n    <g clip-path=\"url(#pcba1dba992)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.762773;stroke:#1f77b4;stroke-opacity:0.762773;\" x=\"213.151047\" xlink:href=\"#C3_0_a2946a6d36\" y=\"215.16581\"/>\n    </g>\n    <g clip-path=\"url(#pcba1dba992)\">\n     <use style=\"fill:#1f77b4;fill-opacity:0.779934;stroke:#1f77b4;stroke-opacity:0.779934;\" x=\"211.189668\" xlink:href=\"#C3_0_a2946a6d36\" y=\"194.466279\"/>\n    </g>\n    <g clip-path=\"url(#pcba1dba992)\">\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"236.119635\" xlink:href=\"#C3_0_a2946a6d36\" y=\"177.059387\"/>\n    </g>\n   </g>\n   <g id=\"text_4\">\n    <!-- c_3 -->\n    <g transform=\"translate(229.119801 134.709886)scale(0.12 -0.12)\">\n     <defs>\n      <path d=\"M 40.578125 39.3125 \nQ 47.65625 37.796875 51.625 33 \nQ 55.609375 28.21875 55.609375 21.1875 \nQ 55.609375 10.40625 48.1875 4.484375 \nQ 40.765625 -1.421875 27.09375 -1.421875 \nQ 22.515625 -1.421875 17.65625 -0.515625 \nQ 12.796875 0.390625 7.625 2.203125 \nL 7.625 11.71875 \nQ 11.71875 9.328125 16.59375 8.109375 \nQ 21.484375 6.890625 26.8125 6.890625 \nQ 36.078125 6.890625 40.9375 10.546875 \nQ 45.796875 14.203125 45.796875 21.1875 \nQ 45.796875 27.640625 41.28125 31.265625 \nQ 36.765625 34.90625 28.71875 34.90625 \nL 20.21875 34.90625 \nL 20.21875 43.015625 \nL 29.109375 43.015625 \nQ 36.375 43.015625 40.234375 45.921875 \nQ 44.09375 48.828125 44.09375 54.296875 \nQ 44.09375 59.90625 40.109375 62.90625 \nQ 36.140625 65.921875 28.71875 65.921875 \nQ 24.65625 65.921875 20.015625 65.03125 \nQ 15.375 64.15625 9.8125 62.3125 \nL 9.8125 71.09375 \nQ 15.4375 72.65625 20.34375 73.4375 \nQ 25.25 74.21875 29.59375 74.21875 \nQ 40.828125 74.21875 47.359375 69.109375 \nQ 53.90625 64.015625 53.90625 55.328125 \nQ 53.90625 49.265625 50.4375 45.09375 \nQ 46.96875 40.921875 40.578125 39.3125 \nz\n\" id=\"DejaVuSans-51\"/>\n     </defs>\n     <use xlink:href=\"#DejaVuSans-99\"/>\n     <use x=\"54.980469\" xlink:href=\"#DejaVuSans-95\"/>\n     <use x=\"104.980469\" xlink:href=\"#DejaVuSans-51\"/>\n    </g>\n   </g>\n  </g>\n </g>\n <defs>\n  <clipPath id=\"pa8abc3a947\">\n   <rect height=\"98.836364\" width=\"98.836364\" x=\"7.2\" y=\"22.10625\"/>\n  </clipPath>\n  <clipPath id=\"p499016b9a9\">\n   <rect height=\"98.836364\" width=\"98.836364\" x=\"189.818182\" y=\"22.10625\"/>\n  </clipPath>\n  <clipPath id=\"p85e40e64a7\">\n   <rect height=\"98.836364\" width=\"98.836364\" x=\"7.2\" y=\"140.709886\"/>\n  </clipPath>\n  <clipPath id=\"pcba1dba992\">\n   <rect height=\"98.836364\" width=\"98.836364\" x=\"189.818182\" y=\"140.709886\"/>\n  </clipPath>\n </defs>\n</svg>\n",
      "text/plain": [
       "<Figure size 432x288 with 4 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "fig = plt.figure()\n",
    "\n",
    "for j in range(m):\n",
    "    ax = fig.add_subplot(2, 2, j+1, projection='3d')\n",
    "    # get centroid positions\n",
    "    X = [c[j][i][0] for i in range(k_)]\n",
    "    Y = [c[j][i][1] for i in range(k_)]\n",
    "    Z = [c[j][i][2] for i in range(k_)]\n",
    "    # plot\n",
    "    ax.scatter(X, Y, Z)\n",
    "    ax.set_title(f\"c_{j}\")\n",
    "    # remove tick values (they're messy)\n",
    "    ax.xaxis.set_ticklabels([])\n",
    "    ax.yaxis.set_ticklabels([])\n",
    "    ax.zaxis.set_ticklabels([])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These are the centroids for each of our subspaces, subvector `u_0` will be mapped to a centroid within subspace `c_0`, `u_1` to `c_1`, etc, etc.\n",
    "\n",
    "Let's go ahead and do this. First, we will define a function to find the nearest centroid using Euclidean distance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "def euclidean(v, u):\n",
    "    distance = sum((x - y) ** 2 for x, y in zip(v, u)) ** .5\n",
    "    return distance\n",
    "\n",
    "def nearest(c_j, u_j):\n",
    "    distance = 9e9\n",
    "    for i in range(k_):\n",
    "        new_dist = euclidean(c_j[i], u_j)\n",
    "        if new_dist < distance:\n",
    "            nearest_idx = i\n",
    "            distance = new_dist\n",
    "    return nearest_idx"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now we calculate the nearest centroids for each subspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 1, 2, 1]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ids = []\n",
    "for j in range(m):\n",
    "    i = nearest(c[j], u[j])\n",
    "    ids.append(i)\n",
    "ids"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we need a way to translate these IDs back into the centroid co-ordinates - well, we already have it - our codebook `c`, when it comes to comparing vectors we don't use the centroid IDs, we use the centroids themselves (our reproduction values)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "q = []\n",
    "for j in range(m):\n",
    "    c_ji = c[j][ids[j]]\n",
    "    q.extend(c_ji)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 7, 7, 6, 3, 6, 7, 4, 4, 3, 9, 6]"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "q"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We typical measure the error between our quantized vectors `q` and the originals `x` using mean squard error (MSE):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "def mse(v, u):\n",
    "    error = sum((x - y) ** 2 for x, y in zip(v, u)) / len(v)\n",
    "    return error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.416666666666667"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mse(x, q)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When using many vectors, we can to minimize the MSE over our original vectors and the centroids by increasing the number of centroids. However this will increase index size and so must be balanced.\n",
    "\n",
    "Lower MSE == more accurate search results *and* higher memory usage.\n",
    "\n",
    "---\n",
    "\n",
    "# Search With PQ\n",
    "\n",
    "Let's move onto building a *searchable* PQ index. For that we need more vectors - which we can get from the *Sift1M* dataset. We're still sticking with our unoptimized but readable implementation for now, so we will only use a small subset of these vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "# now define a function to read the fvecs file format of Sift1M dataset\n",
    "def read_fvecs(fp):\n",
    "    a = np.fromfile(fp, dtype='int32')\n",
    "    d = a[0]\n",
    "    return a.reshape(-1, d + 1)[:, 1:].copy().view('float32')\n",
    "\n",
    "# 1M samples originally, we stick with 1K\n",
    "xb = read_fvecs('../../../data/sift/sift_base.fvecs')[:1000]\n",
    "# queries\n",
    "xq = read_fvecs('../../../data/sift/sift_query.fvecs')[0].reshape(1, -1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1000, 128)"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xb.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1, 128)"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xq.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As before, we must create `m` subvectors (for each vector in `xb` this time)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "16"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m = 8\n",
    "D = xb.shape[1]\n",
    "assert D % m == 0\n",
    "D_ = int(D/m)\n",
    "D_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we'll be working with subvectors with dimensionality `D_` of `16`. We must also set our `k` value, we will use a value of `2048` (or `2^11`) to leave us with `256` subquantizers (`k_`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "256"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "k = 2**11\n",
    "assert k % m == 0\n",
    "k_ = int(k/m)\n",
    "k_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now create all of our centroids - we'll be doing this in Numpy this time so this will look slightly different, but the outcome is the same."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [],
   "source": [
    "c = np.random.randint(1, int(xb.max()+1), (m, k_, D_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(8, 256, 16)"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now have `8` subspaces (set by `m`), each of these correspond to their respective subvector positions `j = 0 -> m`. Within each subspace we have `256` centroids/reproduction values (set by `k_`). Each of these centroids is a `16`-dimensional vector, equivalent to our subvector `u` dimensionalities (set by `D_`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(256, 16)"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c[0].shape  # subspace 0, covering vector dims 0 -> 16"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(256, 16)"
      ]
     },
     "execution_count": 112,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c[1].shape  # subspace 1, covering vector dims 16 -> 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our codebook `c` now contains a big list of all of our centroids, we use the index of each centroid as it's ID - and we will replace each of our datasets `xb` vectors with a sequence of it's nearest centroid IDs.\n",
    "\n",
    "Because each vector is split into `m=8` subvectors, this means each vector can now be represented by `8` ID values.\n",
    "\n",
    "Let's quantize our `xb` using this approach. First we need to split each `xb` vector `x` into subvectors `u` (or `ub` for the full set)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1000, 8, 16)"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ub = xb.reshape(-1, m, D_)\n",
    "ub.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we loop through each subspace, identifying the nearest centroid positions for every single subvector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1000, 8)"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "id_b = np.zeros((xb.shape[0], m), dtype=np.uint8)\n",
    "for row, u in enumerate(ub):\n",
    "    for j in range(m):\n",
    "        # get ID of nearest centroid within current 'j' subspace\n",
    "        idx = nearest(c[j], u[j])\n",
    "        id_b[row, j] = idx\n",
    "id_b.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 48, 110, 218, 172,  29,  87,  75,  43],\n",
       "       [228, 110,  50, 172,  29, 240, 222,  83],\n",
       "       [ 48, 110, 218, 172, 142,  87,  75,  43],\n",
       "       [228, 110, 218, 172,  29, 240, 118,  83],\n",
       "       [ 63, 252, 218, 172, 142,   2,  75,  83]], dtype=uint8)"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "id_b[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We've now converted our vectors into quantized vectors where each 16-dimensional subvector can be represented by a single 8-bit integer. This also has the bonus effect of reducing vector dimensionality from *128 -> 8*.\n",
    "\n",
    "Now, these IDs cannot be used when comparing the distance between a query vector `xq` and the data `id_b` - instead we use the *codebook* `c` to map the subvector IDs in `id_b` to their respective subvector centroids - which we then compare to the equivalent `xq` subvectors.\n",
    "\n",
    "Let's go ahead and do that. First, we create a subvector version of our query vector `xq`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(8, 16)"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "uq = xq.reshape(m, D_)\n",
    "uq.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now we compare this to all of our quantized subvector values (the centroids), which we map using our codebook `c` and ID vectors in `id_b`.\n",
    "\n",
    "We can extract these quantized subvectors like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([121,  65,  36,  59,  75,  19,  21, 132, 101,  57,  22,   2,  62,\n",
       "        69, 112, 145])"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "j = 2  # subspace 2\n",
    "row = 5  # row/sample 5 in our data\n",
    "c[j][id_b[row][j]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's a single quantized subvector, we will loop through and do this for all - and calculate the Euclidean distance with `np.linalg.norm` for *every* subvector - summing them all to get a final value (which does not represent the 'estimated' distance between them)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [],
   "source": [
    "all_dist = []\n",
    "# we will loop through each ID vector in qb\n",
    "for row in range(id_b.shape[0]):\n",
    "    u_dist = 0  # we use this to store all subvector distances\n",
    "    for j in range(m):\n",
    "        # get the quantized subvec value of qb[b]\n",
    "        qb_j = c[j][id_b[row][j]]\n",
    "        # calculate the distance between query subvec and quantized db vec\n",
    "        u_dist += np.linalg.norm(uq[j] - qb_j)\n",
    "    # add total distance between query subvecs and qb subvecs\n",
    "    all_dist.append((u_dist, row))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(1892.7344888809694, 0),\n",
       " (1941.992068055266, 1),\n",
       " (1895.864167006609, 2),\n",
       " (1996.9471078170002, 3),\n",
       " (1942.1256710923853, 4)]"
      ]
     },
     "execution_count": 139,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_dist[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(1532.1243596980867, 408),\n",
       " (1535.5973447572023, 598),\n",
       " (1542.6889938766724, 643),\n",
       " (1552.2655535367062, 270),\n",
       " (1552.5498689085962, 870)]"
      ]
     },
     "execution_count": 140,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_dist.sort()\n",
    "all_dist[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have our nearest neighbors. The top five closest, according to our PQ method are `408`, `598`, `643`, `270`, and `870`. Let's compare the original vectors using Euclidean distance to see if these vectors are genuinely closer to our query vector `xq` than other vectors in `xb`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "524.0475"
      ]
     },
     "execution_count": 164,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# this is our average distance between all vectors\n",
    "np.mean(np.linalg.norm(xb - xq, axis=1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "408: 382.0392761230469\n",
      "598: 334.31719970703125\n",
      "643: 370.2404479980469\n",
      "270: 370.43218994140625\n",
      "870: 373.1098937988281\n",
      "292: 314.4996032714844\n",
      "756: 364.95343017578125\n",
      "172: 365.80596923828125\n",
      "924: 340.1264343261719\n",
      "610: 388.8958740234375\n",
      "779: 379.72491455078125\n",
      "210: 395.8926696777344\n",
      "762: 369.0989074707031\n",
      "882: 282.3154296875\n",
      "467: 406.41357421875\n"
     ]
    }
   ],
   "source": [
    "# and our distances for the top 15...\n",
    "for dist, idx in all_dist[:15]:\n",
    "    print(f\"{idx}: {np.linalg.norm(xb[idx] - xq)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are returning much lower values than the average - but how close are these to the true lowest distances?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "882: 282.32000732421875 | True\n",
      "190: 292.94000244140625 | False\n",
      "816: 311.0299987792969 | False\n",
      "224: 313.8399963378906 | False\n",
      "292: 314.5 | True\n",
      "492: 321.8500061035156 | False\n",
      "146: 326.5299987792969 | False\n",
      "107: 326.80999755859375 | False\n",
      "641: 331.0400085449219 | False\n",
      "598: 334.32000732421875 | True\n",
      "370: 334.5 | False\n",
      "121: 334.9800109863281 | False\n",
      "124: 336.5 | False\n",
      "696: 337.1300048828125 | False\n",
      "924: 340.1300048828125 | True\n"
     ]
    }
   ],
   "source": [
    "dist_arr = np.linalg.norm(xb - xq, axis=1)\n",
    "# get smallest\n",
    "true_lowest = np.argsort(dist_arr)[:15]\n",
    "# get previous IDs of lowest values using our PQ\n",
    "prev_idx = [x[1] for x in all_dist[:15]]\n",
    "for true_idx in true_lowest:\n",
    "    print(f\"{true_idx}: {round(dist_arr[true_idx], 2)} | {true_idx in prev_idx}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So out of the top *15* we have return *4* matches - it's not bad, but not great - which should be expected, we're working with a very basic implementation, we have not trained our clusters, and PQ by itself focuses on reducing memory usage rather than anything else. So all things considered, 4/15 is a reasonable result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 178,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xb.min()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "157.0"
      ]
     },
     "execution_count": 179,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xb.max()"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "a683edd788238e5c64f9fa2e4bdd4387776bc5c6f4f0a84da0685f9a25e421d6"
  },
  "kernelspec": {
   "display_name": "Python 3.8.8 64-bit ('ml': conda)",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
